a bg_z@sddlZddlmZmZmZmZmZmZmZddl Z ddl m Z ddl m Z ddlmZddlmZmZddlmZmZddl mZGd d d e ZGd d d e ZdS) N)AnyDictIteratorListOptionalTupleUnion)CallbackManagerForLLMRun)LLM)GenerationChunk)convert_to_secret_strget_from_dict_or_env)Field SecretStr)ResponsecseZdZUdZeddZeed<eeddZ eed<edddZ eed<edddZ eed <ed dZ e ed <d Zeeeefed <GdddZee dddZeeeefdddZeeeefdddZeedddZed dfdd ZeeeefdddZd/eeeeeefdd d!Zd0eeeefeeeee ed"d#d$Z eed%d&d'Z!ee"e#d%d(d)Z$d1eeeefeeeee%ee"e#d*d+d,Z&d2eeeefeeeee%eed*d-d.Z'Z(S)3 SambaStudiou SambaStudio large language models. Setup: To use, you should have the environment variables ``SAMBASTUDIO_URL`` set with your SambaStudio environment URL. ``SAMBASTUDIO_API_KEY`` set with your SambaStudio endpoint API key. https://sambanova.ai/products/enterprise-ai-platform-sambanova-suite read extra documentation in https://docs.sambanova.ai/sambastudio/latest/index.html Example: .. code-block:: python from langchain_community.llms.sambanova import SambaStudio SambaStudio( sambastudio_url="your-SambaStudio-environment-URL", sambastudio_api_key="your-SambaStudio-API-key, model_kwargs={ "model" : model or expert name (set for Bundle endpoints), "max_tokens" : max number of tokens to generate, "temperature" : model temperature, "top_p" : model top p, "top_k" : model top k, "do_sample" : wether to do sample "process_prompt": wether to process prompt (set for Bundle generic v1 and v2 endpoints) }, ) Key init args — completion params: model: str The name of the model to use, e.g., Meta-Llama-3-70B-Instruct-4096 (set for Bundle endpoints). streaming: bool Whether to use streaming handler when using non streaming methods model_kwargs: dict Extra Key word arguments to pass to the model: max_tokens: int max tokens to generate temperature: float model temperature top_p: float model top p top_k: int model top k do_sample: bool wether to do sample process_prompt: wether to process prompt (set for Bundle generic v1 and v2 endpoints) Key init args — client params: sambastudio_url: str SambaStudio endpoint Url sambastudio_api_key: str SambaStudio endpoint api key Instantiate: .. code-block:: python from langchain_community.llms import SambaStudio llm = SambaStudio=( sambastudio_url = set with your SambaStudio deployed endpoint URL, sambastudio_api_key = set with your SambaStudio deployed endpoint Key, model_kwargs = { "model" : model or expert name (set for Bundle endpoints), "max_tokens" : max number of tokens to generate, "temperature" : model temperature, "top_p" : model top p, "top_k" : model top k, "do_sample" : wether to do sample "process_prompt" : wether to process prompt (set for Bundle generic v1 and v2 endpoints) } ) Invoke: .. code-block:: python prompt = "tell me a joke" response = llm.invoke(prompt) Stream: .. code-block:: python for chunk in llm.stream(prompt): print(chunk, end="", flush=True) Async: .. code-block:: python response = llm.ainvoke(prompt) await response defaultsambastudio_urlsambastudio_api_keyT)rexcludebase_url streaming_urlF streamingN model_kwargsc@seZdZdZdS)zSambaStudio.ConfigTN__name__ __module__ __qualname__Zpopulate_by_namer r p/var/www/html/cobodadashboardai.evdpl.com/venv/lib/python3.9/site-packages/langchain_community/llms/sambanova.pyConfig|sr"returncCsdS)9Return whether this model can be serialized by Langchain.Tr clsr r r!is_lc_serializableszSambaStudio.is_lc_serializablecCs dddS)Nrr)rrr selfr r r! lc_secretsszSambaStudio.lc_secretscCsd|jid|jiS)Return a dictionary of identifying parameters. This information is used by the LangChain callback system, which is used for tracing purposes make it possible to monitor LLMs. rr)rrr)r r r!_identifying_paramsszSambaStudio._identifying_paramscCsdS)zReturn type of llm.zsambastudio-llmr r)r r r! _llm_typeszSambaStudio._llm_typekwargsr$c sTt|dd|d<tt|dd|d<||d\|d<|d<tjfi|dS)'init and validate environment variablesrZSAMBASTUDIO_URLrZSAMBASTUDIO_API_KEYrrN)r r _get_sambastudio_urlssuper__init__r*r0 __class__r r!r4s zSambaStudio.__init__)urlr$cCsZd|vr|}|}n@d|vr,|dd}|}n&|}d|vrJd|d}ntd||fS)a3 Get streaming and non streaming URLs from the given URL Args: url: string with sambastudio base or streaming endpoint url Returns: base_url: string with url to do non streaming calls streaming_url: string with url to do streaming calls chat/completionsstreamzstream/rZgenericzgeneric/streamUnsupported URL)replacejoinsplit ValueError)r*r8rZ stream_urlr r r!r2s  z!SambaStudio._get_sambastudio_urls)stopr$cCsD|dur g}|jpi}|dg|}t|dkr:||d<d|jvrd|vr^|d|d<d|vrx|d|d<d |vr|d |}nd |jvrd|vr|d|d<d|vr|d|d<|}nhd |jvr.d|vr|d|d<d|vr|d|d<d d |D}ntd|jd|S)aD Get the tuning parameters to use when calling the LLM. Args: stop: Stop words to use when generating. Model output is cut off at the first occurrence of any of the stop substrings. Returns: The tuning parameters in the format required by api to use NZstop_sequencesrr9Z select_expertmodelZmax_tokens_to_generate max_tokensprocess_promptapi/v2/predict/genericapi/predict/genericcSs&i|]\}}|t|jt|dqS))typevalue)rFrstr).0kvr r r! sz2SambaStudio._get_tuning_params..r;9only openai, generic v1 and generic v2 APIs are supported)rgetlenrkeyspopitemsr?)r*r@Z _model_kwargsZ_stop_sequencesZ tuning_paramsr r r!_get_tuning_paramssL              zSambaStudio._get_tuning_paramspromptr@rr$c Cst|tr|g}||}d|jvrnd|ddg}||d|}dd|D}d|jd d }nd |jvr|d d rt ddd|ddgd}n|d}d|dg}dd|D}||d}d|ji}nd|jvrr|d d r<|d ddkr2t ddd|ddgd}n|d}n|d}|rV||d}n |g|d}d|ji}nt d|jdt } |r| j |j||dd} n| j |j||d d} | jd krtd!| jd"| jd"| S)#a Performs a post request to the LLM API. Args: prompt: The prompt to pass into the model stop: list of stop tokens streaming: wether to do a streaming call Returns: A request Response object r9userrrolecontent)messagesr:cSsi|]\}}|dur||qSNr rIkeyrGr r r!rLz/SambaStudio._handle_request..Bearer application/json Authorizationz Content-TyperDrCFzsambaverse-conversation-idN)Z message_idrXrY)Zconversation_idrZZitem0)idrGcSsi|]\}}|dur||qSr[r r\r r r!rL2r^)rRparamsr]rErGTrue)instancerd)Z instancesrdr;rMTheadersjsonr:2Sambanova / complete call failed with status code .) isinstancerHrSrrRrget_secret_valuerNridumpsr?requestsSessionpostrr status_code RuntimeErrortext) r*rUr@rrd messages_dictdatarhrR http_sessionresponser r r!_handle_requests~                 zSambaStudio._handle_requestryr$c Csz |}Wn:tyF}z"td|d|jWYd}~n d}~00d|jvrh|dddd}nNd |jvr|d dd d }n.d |jvr|ddd }ntd|jd|S) Process a non streaming response from the api Args: response: A request Response object Returns completion: a string with model generation ;Sambanova /complete call failed couldn't get JSON response response: Nr9choicesrmessagerYrDrRrG completionrEZ predictionsr;rM)ri Exceptionrtrurr?r*ryZ response_dicterr r r!_process_responsehs$      zSambaStudio._process_responsec csz ddl}Wnty&tdYn0d|jvrV||}|D]}|jdkrptd|jd|jdz|jdkrt |jt rt |j}ntd|jd|jd| d rtd|jd|jdt|d dkr|d dd d }nd }t|d}|VWqFtyN}z"td|d|jWYd}~qFd}~00qFn.d|jvr|D]x} z6t | }|ddddd}t|d}|VWn:ty}z td|d| WYd}~n d}~00qjnd|jvrr|D]t} z2t | }|dddd}t|d}|VWn:tyj}z td|d| WYd}~n d}~00qntd|jddS) Process a streaming response from the api Args: response: An iterable request Response object Yields: GenerationChunk: a GenerationChunk with model partial generation rNTcould not import sseclient libraryPlease install it with `pip install sseclient-py`.r9 error_event1Sambanova /complete call failed with status code rl[DONE]errorrdeltarYrru3Error getting content chunk raw streamed response: data: rDresultrRrGZ stream_tokenzline: rE responsesr;rM) sseclient ImportErrorr SSEClienteventseventrtrsrwrmrHriloadsrNrOr r iter_linesr?) r*ryrclientrrwrYgenerated_chunkrliner r r!_process_stream_responses                          z$SambaStudio._process_stream_responserUr@ run_managerr0r$cks:|j||dd}||D]}|r.||j|VqdS)a\Call out to Sambanova's complete endpoint. Args: prompt: The prompt to pass into the model. stop: a list of strings on which the model should stop generating. run_manager: A run manager with callbacks for the LLM. Yields: chunk: GenerationChunk with model partial generation TrNrzrZon_llm_new_tokenrur*rUr@rr0rychunkr r r!_streams  zSambaStudio._streamcKsT|jr6d}|jf|||d|D]}||j7}q"|S|j||dd}||}|S)a Call out to Sambanova's complete endpoint. Args: prompt: The prompt to pass into the model. stop: a list of strings on which the model should stop generating. Returns: result: string with model generation rrUr@rFrrrrurzrr*rUr@rr0rrryr r r!_calls   zSambaStudio._call)N)NF)NN)NN))rrr__doc__rrrH__annotations__rrrrrboolrrrrr" classmethodr(propertyr+r-r.r4rr2rrSrrrzrrr rr rr __classcell__r r r6r!r s^ \"D  f$b   rcseZdZUdZeddZeed<eeddZ eed<eddZ eed<eddZ e ed <ed dZ eed <ed dZeed <eddZeeed<eddZeeed<eddidZeed<GdddZee dddZeeeefdddZeeeefdddZeedddZeddfd d! Zd/e e!eefee!eee e"d"d#d$Z#e"ed%d&d'Z$e"e%e&d%d(d)Z'd0e e!eefee!eee(eed*d+d,Z)d1e e!eefee!eee(ee%e&d*d-d.Z*Z+S)2SambaNovaCloudu SambaNova Cloud large language models. Setup: To use, you should have the environment variables: ``SAMBANOVA_URL`` set with SambaNova Cloud URL. defaults to http://cloud.sambanova.ai/ ``SAMBANOVA_API_KEY`` set with your SambaNova Cloud API Key. Example: .. code-block:: python from langchain_community.llms.sambanova import SambaNovaCloud SambaNovaCloud( sambanova_api_key="your-SambaNovaCloud-API-key, model = model name, max_tokens = max number of tokens to generate, temperature = model temperature, top_p = model top p, top_k = model top k ) Key init args — completion params: model: str The name of the model to use, e.g., Meta-Llama-3-70B-Instruct-4096 (set for CoE endpoints). streaming: bool Whether to use streaming handler when using non streaming methods max_tokens: int max tokens to generate temperature: float model temperature top_p: float model top p top_k: int model top k Key init args — client params: sambanova_url: str SambaNovaCloud Url defaults to http://cloud.sambanova.ai/ sambanova_api_key: str SambaNovaCloud api key Instantiate: .. code-block:: python from langchain_community.llms.sambanova import SambaNovaCloud SambaNovaCloud( sambanova_api_key="your-SambaNovaCloud-API-key, model = model name, max_tokens = max number of tokens to generate, temperature = model temperature, top_p = model top p, top_k = model top k ) Invoke: .. code-block:: python prompt = "tell me a joke" response = llm.invoke(prompt) Stream: .. code-block:: python for chunk in llm.stream(prompt): print(chunk, end="", flush=True) Async: .. code-block:: python response = llm.ainvoke(prompt) await response rr sambanova_urlsambanova_api_keyzMeta-Llama-3.1-8B-InstructrAFrirBgffffff? temperatureNtop_ptop_kZ include_usageTstream_optionsc@seZdZdZdS)zSambaNovaCloud.ConfigTNrr r r r!r"{sr"r#cCsdS)r%Fr r&r r r!r(~sz!SambaNovaCloud.is_lc_serializablecCsddiS)Nrr r)r r r!r+szSambaNovaCloud.lc_secretscCs"|j|j|j|j|j|j|jdS)r,rArrBrrrrrr)r r r!r-sz"SambaNovaCloud._identifying_paramscCsdS)z7Get the type of language model used by this chat model.zsambanovacloud-llmr r)r r r!r.szSambaNovaCloud._llm_typer/c s>t|dddd|d<tt|dd|d<tjfi|dS)r1rZ SAMBANOVA_URLz,https://api.sambanova.ai/v1/chat/completionsrrZSAMBANOVA_API_KEYN)r r r3r4r5r6r r!r4s  zSambaNovaCloud.__init__rTc Cst|tr|g}d|ddg}|||j||j|j|j|jd}dd|D}d|j dd }t }|r|j |j ||d d }n|j |j ||d d }|jd krtd|jd|jd|S)z Performs a post request to the LLM API. Args: prompt: The prompt to pass into the model. stop: list of stop tokens Returns: A request Response object rVrrW)rZr:rBr@rArrrcSsi|]\}}|dur||qSr[r r\r r r!rLr^z2SambaNovaCloud._handle_request..r_r`raTrgFrjrkrl)rmrHrBrArrrrRrrnrprqrrrrsrtru) r*rUr@rrvrwrhrxryr r r!rzsB     zSambaNovaCloud._handle_requestr{c Cs`z |}Wn:tyF}z"td|d|jWYd}~n d}~00|dddd}|S)r|r}r~NrrrrY)rirrtrurr r r!rs  z SambaNovaCloud._process_responsec csJz ddl}Wnty&tdYn0||}|D]}|jdkrdtd|jd|jdz|jdkrt|jt rt |j}ntd|jd|jd| drtd|jd|jdt |d dkr|d dd d }nd }t|d }|VWq:tyB}z"td|d|jWYd}~q:d}~00q:dS)rrNrrrrlrrrrrYrrrr)rrrrrrtrsrwrmrHrirrNrOr r) r*ryrrrrwrYrrr r r!rs\           z'SambaNovaCloud._process_stream_responsercKsT|jr6d}|jf|||d|D]}||j7}q"|S|j||dd}||}|S)Call out to SambaNovaCloud complete endpoint. Args: prompt: The prompt to pass into the model. stop: Optional list of stop words to use when generating. Returns: The string generated by the model. rrFrrrr r r!r1s   zSambaNovaCloud._callcks:|j||dd}||D]}|r.||j|VqdS)rTrNrrr r r!rNs  zSambaNovaCloud._stream)NF)NN)NN),rrrrrrrHrrrrArrrBintrfloatrrrrdictr"rr(rrr+rr-r.r4rrrrzrrr rr rrrr r r6r!rs` @  6<   r)ritypingrrrrrrrrpZ langchain_core.callbacks.managerr Z#langchain_core.language_models.llmsr Zlangchain_core.outputsr Zlangchain_core.utilsr r Zpydanticrrrrrr r r r!s$