a bg.@sdZddlmZmZmZmZmZmZddlm Z m Z ddl m Z m Z mZddlmZmZmZddlmZmZddlmZmZmZddlmZd Zd Zeeeeefeeefd d d dZeeeeee feedddZ!GdddeZ"d S)z Wrapper around Anyscale Endpoint)AnyDictListMappingOptionalSet)AsyncCallbackManagerForLLMRunCallbackManagerForLLMRun) GenerationGenerationChunk LLMResult)convert_to_secret_strget_from_dict_or_envpre_init)Field SecretStr) BaseOpenAIacompletion_with_retrycompletion_with_retry) is_openai_v1z%https://api.endpoints.anyscale.com/v1z$mistralai/Mixtral-8x7B-Instruct-v0.1N)keysresponse token_usagereturncCsN||d}|D]6}||vr0|d|||<q|||d|7<qdS)zUpdate token usage.usageN) intersection)rrrZ _keys_to_use_keyro/var/www/html/cobodadashboardai.evdpl.com/venv/lib/python3.9/site-packages/langchain_community/llms/anyscale.pyupdate_token_usages r)choicespromptsr model_namerc Csdg}t|D]@\}}||}|t|ddt|d|dddgq ||d}t||dS) z2Create the LLMResult from the choices and prompts.messagecontent finish_reasonlogprobs)r%r&)textgeneration_info)rr") generations llm_output) enumerateappendr dictgetr ) r r!rr"r)i_choicer*rrrcreate_llm_result+s  r2cs&eZdZUdZeedZeed<ee ddZ e ed<ee dZ eed<ee dZeed<eed d d Zeeed d dZeeeefd fdd Zeeeefd fdd Zeed ddZdeeeeeeeeedddZdeeeeeee eedddZ!Z"S)AnyscaleaAnyscale large language models. To use, you should have the environment variable ``ANYSCALE_API_KEY``set with your Anyscale Endpoint, or pass it as a named parameter to the constructor. To use with Anyscale Private Endpoint, please also set ``ANYSCALE_BASE_URL``. Example: .. code-block:: python from langchain.llms import Anyscale anyscalellm = Anyscale(anyscale_api_key="ANYSCALE_API_KEY") # To leverage Ray for parallel processing @ray.remote(num_cpus=1) def send_query(llm, text): resp = llm.invoke(text) return resp futures = [send_query.remote(anyscalellm, text) for text in texts] results = ray.get(futures) defaultanyscale_api_baseanyscale_api_keyr")default_factoryprefix_messages)rcCsdS)NFr)clsrrris_lc_serializable\szAnyscale.is_lc_serializable)valuesrcCs,t|ddtd|d<tt|dd|d<t|ddtd|d<zdd l}tr|d|dd }|d s|jfi|j |d <|d s|j fi|j |d <n&|d|d <|d|d<|j |d <Wnt yt dYn0|dr|ddkrt d|dr(|ddkr(t d|S)z?Validate that api key and python package exists in environment.r6ZANYSCALE_API_BASEr4r8ZANYSCALE_API_KEYr"Z MODEL_NAMErN)api_keybase_urlclientZ async_clientZopenai_api_baseZopenai_api_keyzTCould not import openai python package. Please install it with `pip install openai`. streamingnz!Cannot stream results when n > 1.Zbest_ofz'Cannot stream results when best_of > 1.)rDEFAULT_BASE_URLr DEFAULT_MODELopenairget_secret_valuer.ZOpenAIZ completionsZ AsyncOpenAIZ Completion ImportError ValueError)r;r=rFZ client_paramsrrrvalidate_environment`sN         zAnyscale.validate_environmentcsid|jitjS)zGet the identifying parameters.r")r"super_identifying_paramsself __class__rrrLs zAnyscale._identifying_paramscs8d|ji}ts(||j|jdi|tjS)z,Get the parameters used to invoke the model.model)r>Zapi_base)r"rupdater8rGr6rK_invocation_params)rNZ openai_credsrOrrrSszAnyscale._invocation_paramscCsdS)zReturn type of llm.z Anyscale LLMrrMrrr _llm_typeszAnyscale._llm_typeN)r!stop run_managerkwargsrcKsB|j}i||}||||}g}i}hd} d} |D]} |jrt| dkrVtdd} |j| d||fi|D]} | dur| } qt| | 7} qt| dusJ|| j| jr| j dnd| jr| j dnddq8t |f| d|d |}t |t s| }| |d t| ||| s8| d } q8|j||||| d S) atCall out to OpenAI's endpoint with k unique prompts. Args: prompts: The prompts to pass into the model. stop: Optional list of stop words to use when generating. Returns: The full LLM output. Example: .. code-block:: python response = openai.generate(["Tell me a joke."]) Z prompt_tokensZcompletion_tokensZ total_tokensNrC,Cannot stream results with multiple prompts.rr%r&r'r%r&promptrVr system_fingerprintr])rSget_sub_promptsrAlenrI_streamr,r'r(r.r isinstancer-extendrr2rNr!rUrVrWparamsZ sub_promptsr rZ_keysr]Z_promptsZ generationchunkrrrr _generates`        zAnyscale._generatecsD|j}i||}||||}g}i}hd} d} |D]} |jrt| dkrVtdd} |j| d||fi|2z"3dHW} | dur| } qt| | 7} qt6| dusJ|| j| jr| j dnd| jr| j dnddq8t |f| d|d |IdH}t |t s| }| |d t| ||q8|j||||| d S) z:Call out to OpenAI's endpoint async with k unique prompts.rXNrCrYrr%r&rZr[r r^)rSr_rAr`rIZ_astreamr,r'r(r.rrbr-rcrr2rdrrr _ageneratesd        zAnyscale._agenerate)NN)NN)#__name__ __module__ __qualname____doc__rrDr6str__annotations__rr8rEr"listr:r classmethodboolr<rrrJpropertyrrrLrSrTrr r rgrrh __classcell__rrrOrr3AsB 7  R r3)#rltypingrrrrrrZlangchain_core.callbacksrr Zlangchain_core.outputsr r r Zlangchain_core.utilsr rrZpydanticrrZlangchain_community.llms.openairrrZ langchain_community.utils.openairrDrErmrintr2r3rrrrs