a bg0&@sddlZddlZddlZddlZddlZddlmZddlmZddlm Z m Z m Z m Z m Z ddlmZddlmZddlZddlZddlmZddlmZdd lmZdd lmZmZmZmZee Z!Gd d d Z"Gd ddeeZ#Gddde$Z%dS)N)datetime)mktime)AnyDictListLiteralOptional) urlencode)format_date_time) Embeddings)secret_from_env)ndarray) BaseModel ConfigDictField SecretStrc@s$eZdZdZeeeddddZdS)UrlzURL class for parsing the URL.N)hostpathschemareturncCs||_||_||_dSN)rrr)selfrrrru/var/www/html/cobodadashboardai.evdpl.com/venv/lib/python3.9/site-packages/langchain_community/embeddings/sparkllm.py__init__%sz Url.__init____name__ __module__ __qualname____doc__strrrrrrr"src@sXeZdZUdZededdZeed<ededdd dZ e eed <ed ed dd dZ e eed <edd Z e ed<edd Zeded<eddZee e e eeedddZee e eeedddZe e eedddZed,e e e e e d d!d"Zee ed#d$d%Ze eee efd&d'd(Zee e ed)d*d+Z dS)-SparkLLMTextEmbeddingsuSparkLLM embedding model integration. Setup: To use, you should have the environment variable "SPARK_APP_ID","SPARK_API_KEY" and "SPARK_API_SECRET" set your APP_ID, API_KEY and API_SECRET or pass it as a name parameter to the constructor. .. code-block:: bash export SPARK_APP_ID="your-api-id" export SPARK_API_KEY="your-api-key" export SPARK_API_SECRET="your-api-secret" Key init args — completion params: api_key: Optional[str] Automatically inferred from env var `SPARK_API_KEY` if not provided. app_id: Optional[str] Automatically inferred from env var `SPARK_APP_ID` if not provided. api_secret: Optional[str] Automatically inferred from env var `SPARK_API_SECRET` if not provided. base_url: Optional[str] Base URL path for API requests. See full list of supported init args and their descriptions in the params section. Instantiate: .. code-block:: python from langchain_community.embeddings import SparkLLMTextEmbeddings embed = SparkLLMTextEmbeddings( api_key="...", app_id="...", api_secret="...", # other ) Embed single text: .. code-block:: python input_text = "The meaning of life is 42" embed.embed_query(input_text) .. code-block:: python [-0.4912109375, 0.60595703125, 0.658203125, 0.3037109375, 0.6591796875, 0.60302734375, ...] Embed multiple text: .. code-block:: python input_texts = ["This is a test query1.", "This is a test query2."] embed.embed_documents(input_texts) .. code-block:: python [ [-0.1962890625, 0.94677734375, 0.7998046875, -0.1971435546875, 0.445556640625, 0.54638671875, ...], [ -0.44970703125, 0.06585693359375, 0.7421875, -0.474609375, 0.62353515625, 1.0478515625, ...], ] app_idZ SPARK_APP_ID)aliasdefault_factory spark_app_idapi_keyZ SPARK_API_KEYN)default spark_api_key api_secretZSPARK_API_SECRETspark_api_secretz#https://emb-cn-huabei-1.xf-yun.com/base_urlpara)r-querydomainT)Zpopulate_by_name)textsrrc Csd}d}d}|jr|j}|jr,|j}|jr<|j}|j|d||d}g}|D]b}d|ddgi} ||| } tj|| ddid j} | | } | d ur| | qV| d qV|S) aDInternal method to call Spark Embedding API and return embeddings. Args: texts: A list of texts to embed. host: Base URL path for API requests Returns: A list of list of floats representing the embeddings, or list with value None if an error occurs. POST) request_urlmethodr'r*messagesuser)contentZrolez content-typezapplication/json)jsonheadersN) r&Zget_secret_valuer)r+_assemble_ws_auth_url _get_bodyrequestsposttext_parser_messageappendtolist) rr0rr#r'r*urlZ embed_resultr>Z query_contextr7responseZres_arrrrr_embeds6        zSparkLLMTextEmbeddings._embed)r0rcCs|||jS)zPublic method to get embeddings for a list of documents. Args: texts: The list of texts to embed. Returns: A list of embeddings, one for each text, or None if an error occurs. rDr,)rr0rrrembed_documentss z&SparkLLMTextEmbeddings.embed_documents)r>rcCs$||g|j}|dur |dSdS)zPublic method to get embedding for a single query text. Args: text: The text to embed. Returns: Embeddings for the text, or None if an error occurs. NrrE)rr>resultrrr embed_querys z"SparkLLMTextEmbeddings.embed_queryGETr1)r3r4r'r*rcCst|}|j}|j}t}tt|}d ||||} t j | d| dt jd} t| jdd} d|dd| f} t| djdd} ||| d}|d t|S) Nz host: {} date: {} {} {} HTTP/1.1utf-8) digestmod)encodingz:api_key="%s", algorithm="%s", headers="%s", signature="%s"z hmac-sha256zhost date request-line)rdate authorization?)r" _parse_urlrrrnowr r timetupleformathmacnewencodehashlibsha256digestbase64 b64encodedecoder )r3r4r'r*urrrQrMZsignature_originZ signature_shaZsignature_sha_strZauthorization_originrNvaluesrrrr:s.    z,SparkLLMTextEmbeddings._assemble_ws_auth_url)r3rcCsp|d}||dd}|d|d}|d}|dkrHtd|||d}|d|}t|||}|S)Nz:///rzinvalid request url:)indexAssembleHeaderExceptionr)r3ZstidxrrZedidxrr]rrrrPs      z!SparkLLMTextEmbeddings._parse_url)appidr>rc CsF|dddd|jddididd tt|d iid }|S) NZ 39769795890r_)r#uidstatusZembrLutf8)r/featurer5r>rJ)headerZ parameterpayload)r/rZr[r8dumpsrVr\)rrcr>bodyrrrr;s  z SparkLLMTextEmbeddings._get_body)messagercCst|}|dd}|dkr8td|d|dS|ddd}t|}ttj}| d }tj ||d }t |d kr|dd }n|}|SdS) NrhcoderzRequest error: z, rirgr><)dtypei ) r8loadsloggerwarningrZ b64decodenproZfloat32Z newbyteorderZ frombufferlen)rldatarmZ text_baseZ text_datadtr>arrayrrrr?s      z&SparkLLMTextEmbeddings._parser_message)rIr1r1)!rrrr rr r&r__annotations__r)rr+r,r!r/rrZ model_configrfloatrDrFrH staticmethodr:rrPdictrrr;r r?rrrrr",s@ >  "(   r"c@s eZdZdZeddddZdS)rbz3Exception raised for errors in the header assembly.N)msgrcCs ||_dSr)rl)rr}rrrrsz AssembleHeaderException.__init__rrrrrrbsrb)&rZrWrTr8loggingrtimertypingrrrrr urllib.parser Zwsgiref.handlersr numpyrtr<Zlangchain_core.embeddingsr Zlangchain_core.utilsr r Zpydanticrrrr getLoggerrrqrr" Exceptionrbrrrrs(         e