a bgf0@sddlmZddlZddlmZddlmZmZmZm Z m Z m Z ddl m Z ddlmZddlmZddlmZmZdd lmZdd lmZmZeeZGd d d eZdS) ) annotationsN)Path)AnyDictIteratorListOptionalUnion)CallbackManagerForLLMRun)LLM)GenerationChunk)get_pydantic_field_namespre_init)_build_model_kwargs)Fieldmodel_validatorc@seZdZUdZdZded<ded<dZded<dZded <ed d d Z d ed <eddd Z d ed<eddd Z d ed<eddd Z ded<eddd Z ded<eddd Zded<eddd Zded<eddd Zded<eddd Zded<eddd Zded<edZded<dZded<d Zd!ed"<d#Zd!ed$<edZded%<dZd&ed'<gZd(ed)<d*Zd!ed+<d,Zded-<d.Zded/<dZd&ed0<d1Zd2ed3<d4Zd2ed5<ee d6Z!d7ed8<dZ"ded9<dZ#d:ed;<dZ$d<e&d?d?d@dAdBZ'e(dCdDe)d7dd@dEdFZ*e+d7dGdHdIZ,e+d7dGdJdKZ-e+ddGdLdMZ.d[d(d7dNdOdPZ/d\dd(dQdddRdSdTZ0d]dd(dQddUdRdVdWZ1dd dXdYdZZ2dS)^LlamaCppallama.cpp model. To use, you should have the llama-cpp-python library installed, and provide the path to the Llama model as a named parameter to the constructor. Check out: https://github.com/abetlen/llama-cpp-python Example: .. code-block:: python from langchain_community.llms import LlamaCpp llm = LlamaCpp(model_path="/path/to/llama/model") Nrclientstr model_pathz Optional[str] lora_base lora_pathin_ctx)aliasintn_partsseedTf16_kvboolF logits_all vocab_only use_mlock n_threadsz Optional[int]n_batch n_gpu_layerssuffix max_tokensg?zOptional[float] temperaturegffffff?top_plogprobszOptional[bool]echozOptional[List[str]]stopg?repeat_penalty(top_k@last_n_tokens_sizeuse_mmapg?floatrope_freq_scaleg@rope_freq_base)default_factoryzDict[str, Any] model_kwargs streamingzOptional[Union[str, Path]] grammar_pathzOptional[Union[str, Any]]grammarverboser)valuesreturnc sLzddlm}m}Wnty.tdYn0d}gd}fdd|D}dd urjd|d<|d z||fi|d <Wn8ty}z td |d |WYd }~n d }~00drdrd}d} td|d| dnBtdtr*| dd<ndrH| dd<nS)z4Validate that llama-cpp-python library is installed.r)Llama LlamaGrammarzCould not import llama-cpp-python library. Please install the llama-cpp-python library to use this embedding model: pip install llama-cpp-pythonr)r6r7rrrrrrr r!r"r#r%r4r3r=csi|]}||qSrB).0kr>rBo/var/www/html/cobodadashboardai.evdpl.com/venv/lib/python3.9/site-packages/langchain_community/llms/llamacpp.py z1LlamaCpp.validate_environment..r&Nr9rz&Could not load Llama model from path: z. Received error r<r;zCCan only pass in one of grammar and grammar_path. Received grammar=z and grammar_path=.) Z llama_cppr@rA ImportErrorupdate Exception ValueError isinstancerZ from_string from_file) clsr>r@rArZmodel_param_namesZ model_paramser<r;rBrErFvalidate_environmentsF     zLlamaCpp.validate_environmentbefore)modecCst|}t||}|S)z>Build extra kwargs from additional params that were passed in.)r r)rPr>Zall_required_field_namesrBrBrFbuild_model_kwargss zLlamaCpp.build_model_kwargs)r?c Cs>|j|j|j|j|j|j|j|j|jd }|j r:|j |d<|S)z1Get the default parameters for calling llama_cpp.) r'r)r*r+r,r-stop_sequencesr/r1r<) r'r)r*r+r,r-r.r/r1r<)selfparamsrBrBrF_default_paramss  zLlamaCpp._default_paramscCsid|ji|jS)zGet the identifying parameters.r)rrYrWrBrBrF_identifying_paramsszLlamaCpp._identifying_paramscCsdS)zReturn type of llm.ZllamacpprBrZrBrBrF _llm_typeszLlamaCpp._llm_type)r.r?cCs<|jr|durtd|j}|d|jp2|p2g|d<|S)a Performs sanity check, preparing parameters in format needed by llama_cpp. Args: stop (Optional[List[str]]): List of stop sequences for llama_cpp. Returns: Dictionary containing the combined parameters. Nz2`stop` found in both the input and default params.rVr.)r.rMrYpop)rWr.rXrBrBrF_get_parameterss  zLlamaCpp._get_parametersz"Optional[CallbackManagerForLLMRun])promptr. run_managerkwargsr?c Kst|jr6d}|jf|||d|D]}||j7}q"|S||}i||}|jfd|i|}|dddSdS)aCall the Llama model and return the output. Args: prompt: The prompt to use for generation. stop: A list of strings to stop generation when encountered. Returns: The generated text. Example: .. code-block:: python from langchain_community.llms import LlamaCpp llm = LlamaCpp(model_path="/path/to/local/llama/model.bin") llm.invoke("This is a prompt.") )r_r.r`r_choicesrtextN)r:_streamrdr^r) rWr_r.r`raZcombined_text_outputchunkrXresultrBrBrF_calls    zLlamaCpp._callzIterator[GenerationChunk]c ksi|||}|jf|dd|}|D]R}|dddd}t|dddd|id} |rx|j| j|j|d | Vq,dS) a\Yields results objects as they are generated in real time. It also calls the callback manager's on_llm_new_token event with similar parameters to the OpenAI LLM class method of the same name. Args: prompt: The prompts to pass into the model. stop: Optional list of stop words to use when generating. Returns: A generator representing the stream of tokens being generated. Yields: A dictionary like objects containing a string token and metadata. See llama-cpp-python docs and below for more. Example: .. code-block:: python from langchain_community.llms import LlamaCpp llm = LlamaCpp( model_path="/path/to/local/model.bin", temperature = 0.5 ) for chunk in llm.stream("Ask 'Hi, how are you?' like a pirate:'", stop=["'"," "]): result = chunk["choices"][0] print(result["text"], end='', flush=True) # noqa: T201 T)r_streamrcrr,Nrd)rdZgeneration_info)tokenr=Z log_probs)r^rgetr Zon_llm_new_tokenrdr=) rWr_r.r`rarXrgpartr,rfrBrBrFre,s% zLlamaCpp._stream)rdr?cCs|j|d}t|S)Nzutf-8)rtokenizeencodelen)rWrdZtokenized_textrBrBrFget_num_tokens_szLlamaCpp.get_num_tokens)N)NN)NN)3__name__ __module__ __qualname____doc__r__annotations__rrrrrrrr r!r"r#r%r&r'r)r*r+r,r-r.r/r1r3r4r6r7dictr9r:r;r<r=rrRr classmethodrUpropertyrYr[r\r^rhrerprBrBrBrFrsh                   <-3r) __future__rloggingpathlibrtypingrrrrrr Zlangchain_core.callbacksr Z#langchain_core.language_models.llmsr Zlangchain_core.outputsr Zlangchain_core.utilsr rZlangchain_core.utils.utilsrZpydanticrr getLoggerrqloggerrrBrBrBrFs