a bg"@sddlmZddlmZmZmZmZmZmZm Z ddlmZddl m Z ddlmZddl m Z mZddlmZddlmZddlmZddlmZddlmZddlmZGdd d eZd S) )pre_init)Any AsyncIteratorDictIteratorListOptionalUnion)root_validator)AsyncCallbackManagerForLLMRunCallbackManagerForLLMRun)LLM)enforce_stop_tokens)GenerationChunkc@s4eZdZUdZeed<eed<dZee eefed<dZ e dee fed<dZ e ed<ee eefd d d Zeed d d Zee e dddZdeeeeeeeedddZdeeeeeeeedddZdeeeeeeeeedddZdeeeeeeeeedddZdS) DeepSparseaHNeural Magic DeepSparse LLM interface. To use, you should have the ``deepsparse`` or ``deepsparse-nightly`` python package installed. See https://github.com/neuralmagic/deepsparse This interface let's you deploy optimized LLMs straight from the [SparseZoo](https://sparsezoo.neuralmagic.com/?useCase=text_generation) Example: .. code-block:: python from langchain_community.llms import DeepSparse llm = DeepSparse(model="zoo:nlg/text_generation/codegen_mono-350m/pytorch/huggingface/bigpython_bigquery_thepile/base_quant-none") pipelinemodelNmodel_configurationgeneration_configF streaming)returncCs|j|j|j|jdS)zGet the identifying parameters.)r model_configrr)rrrrselfrq/var/www/html/cobodadashboardai.evdpl.com/venv/lib/python3.9/site-packages/langchain_community/llms/deepsparse.py_identifying_params1s zDeepSparse._identifying_paramscCsdS)zReturn type of llm. deepsparserrrrr _llm_type;szDeepSparse._llm_type)valuesrcCsZzddlm}Wnty*tdYn0|dp6i}|jfd|dd||d<|S) z2Validate that ``deepsparse`` package is installed.r)Pipelinez[Could not import `deepsparse` package. Please install it with `pip install deepsparse[llm]`rZtext_generationr)taskZ model_pathr)rr ImportErrorcreate)clsrr rrrrvalidate_environment@s    zDeepSparse.validate_environment)promptstop run_managerkwargsrcKsl|jr8d}|jf|||d|D]}||j7}q"|}n|jfd|i|jjdj}|durht||}|S)Generate text from a prompt. Args: prompt: The prompt to generate text from. stop: A list of strings to stop generation when encountered. Returns: The generated text. Example: .. code-block:: python from langchain_community.llms import DeepSparse llm = DeepSparse(model="zoo:nlg/text_generation/codegen_mono-350m/pytorch/huggingface/bigpython_bigquery_thepile/base_quant-none") llm.invoke("Tell me a joke.") r&r'r( sequencesrN)r_streamtextrr generationsrrr&r'r(r)Zcombined_outputchunkr/rrr_callTs    zDeepSparse._callcsv|jrBd}|jf|||d|2z3dHW}||j7}q"6|}n|jfd|i|jjdj}|durrt||}|S)r*r+r,Nr-r)r_astreamr/rrr0rr1rrr_acallzs  zDeepSparse._acallcksP|jf|dd|j}|D].}t|jdjd}|rD|j|jd|VqdSaYields results objects as they are generated in real time. It also calls the callback manager's on_llm_new_token event with similar parameters to the OpenAI LLM class method of the same name. Args: prompt: The prompt to pass into the model. stop: Optional list of stop words to use when generating. Returns: A generator representing the stream of tokens being generated. Yields: A dictionary like object containing a string token. Example: .. code-block:: python from langchain_community.llms import DeepSparse llm = DeepSparse( model="zoo:nlg/text_generation/codegen_mono-350m/pytorch/huggingface/bigpython_bigquery_thepile/base_quant-none", streaming=True ) for chunk in llm.stream("Tell me a joke", stop=["'"," "]): print(chunk, end='', flush=True) # noqa: T201 T)r-rr)r/)tokenNrrrr0r/Zon_llm_new_tokenrr&r'r(r)Z inferencer7r2rrrr.szDeepSparse._streamcKsV|jf|dd|j}|D]4}t|jdjd}|rJ|j|jdIdH|VqdSr6r8r9rrrr4szDeepSparse._astream)NN)NN)NN)NN)__name__ __module__ __qualname____doc__r__annotations__strrrrrr rboolpropertyrrrr%rr r3r r5rrr.rr4rrrrrs`      )  )  ) rN)Zlangchain_core.utilsrtypingrrrrrrr Zpydanticr Zlangchain_core.callbacksr r Z#langchain_core.language_models.llmsr Zlangchain_community.llms.utilsrZlangchain_core.outputsrrrrrrs $