a bg(@sddlmZddlZddlmZddlmZmZmZm Z m Z ddl Z ddl m Z ddlmZddlmZddlmZdd lmZGd d d eZdS) ) annotationsN)StringIO)AnyDictIteratorListOptional)CallbackManagerForLLMRun)LLM)GenerationChunk)get_pydantic_field_names) ConfigDictc@seZdZUdZdZded<dZded<dZd ed <d Zd ed <dZ ded<dZ d ed<dZ ded<dZ ded<d Z d ed<dZd ed<dZded<dZded<dZded<dZd ed <d!Zd ed"<d#Zded$<d#Zded%<dZd ed&<d'Zded(<d)Zded*<ed+d,Zedd-d.d/Zed0d-d1d2Zed3d-d4d5ZdEd6d7d3d8d9d:ZdFdd6d;d7dd<d=d>Z dGdd6d;d7d?d<d@dAZ!dddBdCdDZ"dS)H LlamafileaOLlamafile lets you distribute and run large language models with a single file. To get started, see: https://github.com/Mozilla-Ocho/llamafile To use this class, you will need to first: 1. Download a llamafile. 2. Make the downloaded file executable: `chmod +x path/to/model.llamafile` 3. Start the llamafile in server mode: `./path/to/model.llamafile --server --nobrowser` Example: .. code-block:: python from langchain_community.llms import Llamafile llm = Llamafile() llm.invoke("Tell me a joke.") zhttp://localhost:8080strbase_urlNz Optional[int]request_timeoutFbool streamingintseedg?float temperature(top_kgffffff?top_pg?min_p n_predictrn_keepg?tfs_z typical_pg?repeat_penalty@ repeat_last_nT penalize_nlgpresence_penaltyfrequency_penaltymirostatg@ mirostat_taug? mirostat_etaZforbid)extra)returncCsdS)NZ llamafile)selfr,r,p/var/www/html/cobodadashboardai.evdpl.com/venv/lib/python3.9/site-packages/langchain_community/llms/llamafile.py _llm_typexszLlamafile._llm_typez List[str]cs$gdfddt|jD}|S)N) rcacheZcallback_manager callbacksmetadatanamerrtagsverboseZcustom_get_token_idscsg|]}|vr|qSr,r,).0kZ ignore_keysr,r. sz/Llamafile._param_fieldnames..)r __class__)r-attrsr,r8r._param_fieldnames|s  zLlamafile._param_fieldnameszDict[str, Any]cCs"i}|jD]}t||||<q |S)N)r<getattr)r-params fieldnamer,r,r._default_paramss zLlamafile._default_paramszOptional[List[str]]r)stopkwargsr+cKsV|j}|D]\}}||vr|||<q|durDt|dkrD||d<|jrRd|d<|S)NrrATstream)r@itemslenr)r-rArBr>r7vr,r,r._get_parameterss zLlamafile._get_parametersz"Optional[CallbackManagerForLLMRun])promptrA run_managerrBr+c Ks|jrbtB}|j|f||d|D]}||jq&|}Wdn1sT0Y|S|jfd|i|}d|i|} z&tj|j dddi| d|j d } Wn,tj j ytj d |j d Yn0| d | _| d }|SdS)aRequest prompt completion from the llamafile server and return the output. Args: prompt: The prompt to use for generation. stop: A list of strings to stop generation when encountered. run_manager: **kwargs: Any additional options to pass as part of the generation request. Returns: The string generated by the model. )rArINrArH /completion Content-Typeapplication/jsonFurlheadersjsonrCtimeoutTCould not connect to Llamafile server. Please make sure that a server is running at .zutf-8content)rr_streamwritetextgetvaluerGrequestspostrr exceptionsConnectionErrorraise_for_statusencodingrP) r-rHrArIrBZbuffchunkrWr>payloadresponser,r,r._callsB &     zLlamafile._callzIterator[GenerationChunk]c ks|jfd|i|}d|vr$d|d<d|i|}z&tj|jdddi|d|jd}Wn,tjjytjd |jd Yn0d |_|jdd D]0}| |} t | d } |r|j | j d| VqdS)a"Yields results objects as they are generated in real time. It also calls the callback manager's on_llm_new_token event with similar parameters to the OpenAI LLM class method of the same name. Args: prompt: The prompts to pass into the model. stop: Optional list of stop words to use when generating. run_manager: **kwargs: Any additional options to pass as part of the generation request. Returns: A generator representing the stream of tokens being generated. Yields: Dictionary-like objects each containing a token Example: .. code-block:: python from langchain_community.llms import Llamafile llm = Llamafile( temperature = 0.0 ) for chunk in llm.stream("Ask 'Hi, how are you?' like a pirate:'", stop=["'"," "]): result = chunk["choices"][0] print(result["text"], end='', flush=True) rArCTrHrJrKrLrMrRrSutf8)decode_unicode)rW)tokenN) rGrYrZrrr[r\r^ iter_lines_get_chunk_contentr Zon_llm_new_tokenrW) r-rHrArIrBr>r`raZ raw_chunkrTr_r,r,r.rUs4&      zLlamafile._stream)r_r+cCs.|dr&|d}t|}|dS|SdS)zWhen streaming is turned on, llamafile server returns lines like: 'data: {"content":" They","multimodal":true,"slot_id":0,"stop":false}' Here, we convert this to a dict and return the value of the 'content' field zdata:zdata: rTN) startswithlstriprPloads)r-r_cleaneddatar,r,r.rg1s   zLlamafile._get_chunk_content)N)NN)NN)#__name__ __module__ __qualname____doc__r__annotations__rrrrrrrrrrr r!r#r$r%r&r'r(r)r Z model_configpropertyr/r<r@rGrbrUrgr,r,r,r.rsN                     ?Fr) __future__rrPiortypingrrrrrrYZ langchain_core.callbacks.managerr Z#langchain_core.language_models.llmsr Zlangchain_core.outputsr Zlangchain_core.utilsr Zpydanticr rr,r,r,r.s