a bg $@sdZddlmZmZmZmZmZmZmZm Z m Z m Z ddl m Z mZddlmZddlmZddlmZmZmZmZmZddlmZmZmZmZddlmZdd l m!Z!dd l"m#Z#dd l$m%Z%d Z&Gd ddeZ'dS)zMLX Chat Wrapper.) AnyCallableDictIteratorListLiteralOptionalSequenceTypeUnion)AsyncCallbackManagerForLLMRunCallbackManagerForLLMRun)LanguageModelInput) BaseChatModel) AIMessageAIMessageChunk BaseMessage HumanMessage SystemMessage)ChatGenerationChatGenerationChunk ChatResult LLMResult)Runnable)BaseToolconvert_to_openai_tool) MLXPipelinez4You are a helpful, respectful, and honest assistant.cspeZdZUdZeed<eedZeed<dZ e ed<e dfdd Z d#e e ee eeee ed d d Zd$e e ee eeee ed d dZd%e e eeeedddZe edddZeeedddZeedddZd&e e ee eeee eed ddZ dde!e"e#ee fe$e%e&fee"eee'defe e(e)e fd fd!d"Z*Z+S)'ChatMLXaMLX chat models. Works with `MLXPipeline` LLM. To use, you should have the ``mlx-lm`` python package installed. Example: .. code-block:: python from langchain_community.chat_models import chatMLX from langchain_community.llms import MLXPipeline llm = MLXPipeline.from_model_id( model_id="mlx-community/quantized-gemma-2b-it", ) chat = chatMLX(llm=llm) llmcontentsystem_messageN tokenizer)kwargsc s tjfi||jj|_dS)N)super__init__rr#)selfr$ __class__q/var/www/html/cobodadashboardai.evdpl.com/venv/lib/python3.9/site-packages/langchain_community/chat_models/mlx.pyr&DszChatMLX.__init__)messagesstop run_managerr$returncKs0||}|jjf|g||d|}||SN)Zpromptsr-r.)_to_chat_promptr _generate_to_chat_resultr'r,r-r.r$ llm_input llm_resultr*r*r+r2Hs zChatMLX._generatecs6||}|jjf|g||d|IdH}||Sr0)r1r _agenerater3r4r*r*r+r7Us  zChatMLX._agenerateF)r,tokenizereturn_tensorsr/csH|s tdt|dts"tdfdd|D}jj||d|dS)zHConvert a list of messages into a prompt format expected by wrapped LLM.z+At least one HumanMessage must be provided!z$Last message must be a HumanMessage!csg|]}|qSr*)_to_chatml_format).0mr'r*r+ oz+ChatMLX._to_chat_prompt..T)r8Zadd_generation_promptr9) ValueError isinstancerr#Zapply_chat_template)r'r,r8r9Zmessages_dictsr*r>r+r1bszChatMLX._to_chat_prompt)messager/cCsNt|trd}n2t|tr d}n"t|tr0d}ntdt|||jdS)z+Convert LangChain message to ChatML format.systemZ assistantuserzUnknown message type: )roler!)rBrrrrAtyper!)r'rCrFr*r*r+r;ws   zChatMLX._to_chatml_format)r6r/cCsBg}|jdD]$}tt|jd|jd}||qt||jdS)Nrr )rCgeneration_info) generations llm_output)rIrrtextrHappendrrJ)r6Zchat_generationsgZchat_generationr*r*r+r3s zChatMLX._to_chat_result)r/cCsdS)Nzmlx-chat-wrapperr*r>r*r*r+ _llm_typeszChatMLX._llm_typecksfddlm}ddlm}zddlm}ddlm}WntyNtdYn0|d|jj}|dd}|dd} |d d} |d d} |j|d d d } | | d} |j j }t || |jj || | dt| D]\\}}}d}t|ts|j |}n |j |}|r@tt|dd}|r:|j||d|V||ksZ|dur||vrqbqdS)Nr) generate_stepzTCould not import mlx_lm python package. Please install it with `pip install mlx_lm`. model_kwargstempgZ max_tokensdrepetition_penaltyrepetition_context_sizeTnp)r8r9)rQrSrTr )rC)chunk)Zmlx.corecoreZ mlx_lm.utilsrO ImportErrorgetrZpipeline_kwargsr1arrayr# eos_token_idzipmodelrangerBintdecodeitemrrZon_llm_new_token)r'r,r-r.r$ZmxrOrPrQZmax_new_tokensrSrTr5Z prompt_tokensr[tokenZprobnrKrVr*r*r+_streamsT          zChatMLX._stream) tool_choiceautonone)toolsrer$r/c sdd|D}|dur|rt|dkr:tdt|dt|tr\|dvrdd |id }nlt|trp|d }nXt|tr|d dd |dd krtd |d |d dd dntd|||d<tjfd|i|S)a*Bind tool-like objects to this chat model. Assumes model is compatible with OpenAI tool-calling API. Args: tools: A list of tool definitions to bind to this chat model. Supports any tool definition handled by :meth:`langchain_core.utils.function_calling.convert_to_openai_tool`. tool_choice: Which tool to require the model to call. Must be the name of the single provided function or "auto" to automatically determine which function to call (if any), or a dict of the form: {"type": "function", "function": {"name": <>}}. **kwargs: Any additional parameters to pass to the :class:`~langchain.runnable.Runnable` constructor. cSsg|] }t|qSr*r)r<Ztoolr*r*r+r?r@z&ChatMLX.bind_tools..NzKWhen specifying `tool_choice`, you must provide exactly one tool. Received z tools.rffunctionname)rGrkrz Tool choice z/ was specified, but the only provided tool was .zEUnrecognized tool_choice type. Expected str, bool or dict. Received: reri)lenrArBstrbooldictr%bind)r'rirer$Zformatted_toolsr(r*r+ bind_toolss@       zChatMLX.bind_tools)NN)NN)FN)NN),__name__ __module__ __qualname____doc__r__annotations__rDEFAULT_SYSTEM_PROMPTr"r#rr&rrrror rr2r r7rpr1rqr; staticmethodrr3propertyrNrrrdr r rr rrrrrrs __classcell__r*r*r(r+r,sh          C rN)(rwtypingrrrrrrrr r r Z langchain_core.callbacks.managerr r Zlangchain_core.language_modelsrZ*langchain_core.language_models.chat_modelsrZlangchain_core.messagesrrrrrZlangchain_core.outputsrrrrZlangchain_core.runnablesrZlangchain_core.toolsrZ%langchain_core.utils.function_callingrZ%langchain_community.llms.mlx_pipelinerryrr*r*r*r+s0