a bg'8@sddlZddlZddlmZmZmZmZmZmZddl Z ddl m Z m Z ddl mZddlmZddlmZeeZGdddeZdS) N)Any AsyncIteratorDictIteratorListOptional)AsyncCallbackManagerForLLMRunCallbackManagerForLLMRun)LLM)GenerationChunk)Fieldc@seZdZUdZeed<dZeeed<dZee ed<e ddd Z e ed<d Z eeed <d Zeeed <dZeeed<dZeeed<dZeeed<dZeeed<dZeeed<dZee ed<dZee ed<dZee ed<dZeeed<dZeeed<e ddd Ze ed<e ddd Ze ed<e dd d Ze ed <d!Zee ed"<e dd#d Ze ed#<e dd$d Ze ed$<gZ ee!eed%<dZ"e ed&<e#e$ee%fd'd(d)Z&e#e$ee%fd'd*d+Z'e#ed'd,d-Z(d:ee!ee$ee%fd.d/d0Z)d;eee!eee*e%ed1d2d3Z+deee!eee,e%e1e/d1d8d9Z2dS)?TextGenazText generation models from WebUI. To use, you should have the text-generation-webui installed, a model loaded, and --api added as a command-line option. Suggested installation, use one-click installer for your OS: https://github.com/oobabooga/text-generation-webui#one-click-installers Parameters below taken from text-generation-webui api example: https://github.com/oobabooga/text-generation-webui/blob/main/api-examples/api-example.py Example: .. code-block:: python from langchain_community.llms import TextGen llm = TextGen(model_url="http://localhost:8500") model_urlNpresetmax_new_tokensT do_sample)aliasg? temperatureg?top_p typical_prepsilon_cutoff eta_cutoffgzG?repetition_penalty(top_k min_lengthno_repeat_ngram_size num_beams penalty_alphalength_penaltyFearly_stoppingseed add_bos_tokenitruncation_length ban_eos_tokenskip_special_tokensstopping_strings streaming)returncCsZ|j|j|j|j|j|j|j|j|j|j |j |j |j |j |j|j|j|j|j|j|jdS)z/Get the default parameters for calling textgen.rrrrrrrrrrrrr r!r"r$r%r&r'r(r)r,selfr/n/var/www/html/cobodadashboardai.evdpl.com/venv/lib/python3.9/site-packages/langchain_community/llms/textgen.py_default_paramsvs,zTextGen._default_paramscCsid|ji|jS)zGet the identifying parameters.r)rr1r-r/r/r0_identifying_paramsszTextGen._identifying_paramscCsdS)zReturn type of llm.Ztextgenr/r-r/r/r0 _llm_typeszTextGen._llm_type)stopr+cCsH|jr|durtd|jdur(|j}n d|ji}|jp>|p>g|d<|S)a Performs sanity check, preparing parameters in format needed by textgen. Args: stop (Optional[List[str]]): List of stop sequences for textgen. Returns: Dictionary containing the combined parameters. Nz2`stop` found in both the input and default params.rr))r) ValueErrorrr1)r.r4paramsr/r/r0_get_parameterss   zTextGen._get_parameters)promptr4 run_managerkwargsr+c Ks|jr8d}|jf|||d|D]}||j7}q"|}nf|jd}||} | } || d<tj|| d} | jdkr| ddd }nt d | d}|S) Call the textgen web API and return the output. Args: prompt: The prompt to use for generation. stop: A list of strings to stop generation when encountered. Returns: The generated text. Example: .. code-block:: python from langchain_community.llms import TextGen llm = TextGen(model_url="http://localhost:5000") llm.invoke("Write a story about llamas.") r8r4r9/api/v1/generater8jsonresultsrtextERROR: response: ) r*_streamrCrr7copyrequestspost status_coder@print r.r8r4r9r:Zcombined_text_outputchunkresulturlr6requestresponser/r/r0_calls&     z TextGen._callc s|jrBd}|jf|||d|2z3dHW}||j7}q"6|}nf|jd}||} | } || d<tj|| d} | jdkr| dd d }nt d | d}|S) r;r<r=Nr>r8r?rArBrrCrD) r*_astreamrCrr7rFrGrHrIr@rJrKr/r/r0_acalls&   zTextGen._acallc ksz ddl}Wnty&tdYn0i|||}|jd}|}||d<|} | || t || } t | } | ddkrt | ddd } |r|j | jd | Vqx| dd krx| dSqxdS aYields results objects as they are generated in real time. It also calls the callback manager's on_llm_new_token event with similar parameters to the OpenAI LLM class method of the same name. Args: prompt: The prompts to pass into the model. stop: Optional list of stop words to use when generating. Returns: A generator representing the stream of tokens being generated. Yields: A dictionary like objects containing a string token and metadata. See text-generation-webui docs and below for more. Example: .. code-block:: python from langchain_community.llms import TextGen llm = TextGen( model_url = "ws://localhost:5005" streaming=True ) for chunk in llm.stream("Ask 'Hi, how are you?' like a pirate:'", stop=["'"," "]): print(chunk, end='', flush=True) # noqa: T201 rNz9The `websocket-client` package is required for streaming.z/api/v1/streamr8eventZ text_streamrC)rCZgeneration_info)tokenZ stream_end websocket ImportErrorr7rrFZ WebSocketconnectsendr@dumpsrecvloadsr Zon_llm_new_tokenrCclose r.r8r4r9r:rXr6rNrOZwebsocket_clientrMrLr/r/r0rEs4$        zTextGen._streamc Ksz ddl}Wnty&tdYn0i|||}|jd}|}||d<|} | || t || } t | } | ddkrt | ddd } |r|j | jd IdH| Vqx| dd krx| dSqxdSrTrWr`r/r/r0rRYs4$        zTextGen._astream)N)NN)NN)NN)NN)3__name__ __module__ __qualname____doc__str__annotations__rrrintr rboolrfloatrrrrrrrrrr r!r"r$r%r&r'r(r)rr*propertyrrr1r2r3r7r rQrrSrr rErrRr/r/r/r0r s  "  1  1  K r )r@loggingtypingrrrrrrrGZlangchain_core.callbacksrr Z#langchain_core.language_models.llmsr Zlangchain_core.outputsr Zpydanticr getLoggerraloggerr r/r/r/r0s