a bg<'@sfddlZddlmZmZmZmZddlmZddlm Z ddl m Z dZ e eZGddde ZdS) N)AnyListMappingOptional)CallbackManagerForLLMRun)LLM) ConfigDictZgpt2c @sZeZdZUdZeZeed<dZe e ed<dZ e ed<dZ e ed<dZeed<ed d Zed"dddd ee e e eee ee ed d dZed#ddee e e ee edddZed$ee eee eee e e e e dddZee eee e e dddZeeee fdddZeedddZd%ee eee ee edd d!ZdS)&IpexLLMzIpexLLM model. Example: .. code-block:: python from langchain_community.llms import IpexLLM llm = IpexLLM.from_model_id(model_id="THUDM/chatglm-6b") model_idN model_kwargsmodel tokenizerT streamingZforbid)extra) tokenizer_id load_in_4bitload_in_low_bit)r r rrrkwargsreturnc Ks|j||d||||dS)a Construct object from model_id Args: model_id: Path for the huggingface repo id to be downloaded or the huggingface checkpoint folder. tokenizer_id: Path for the huggingface repo id to be downloaded or the huggingface checkpoint folder which contains the tokenizer. load_in_4bit: "Whether to load model in 4bit. Unused if `load_in_low_bit` is not None. load_in_low_bit: Which low bit precisions to use when loading model. Example values: 'sym_int4', 'asym_int4', 'fp4', 'nf4', 'fp8', etc. Overrides `load_in_4bit` if specified. model_kwargs: Keyword arguments to pass to the model and tokenizer. kwargs: Extra arguments to pass to the model and tokenizer. Returns: An object of IpexLLM. Fr r low_bit_modelrrr r _load_model)clsr r rrrrro/var/www/html/cobodadashboardai.evdpl.com/venv/lib/python3.9/site-packages/langchain_community/llms/ipex_llm.py from_model_id's zIpexLLM.from_model_id)r)r r rrrc Ks|j||ddd||dS)a Construct low_bit object from model_id Args: model_id: Path for the ipex-llm transformers low-bit model folder. tokenizer_id: Path for the huggingface repo id or local model folder which contains the tokenizer. model_kwargs: Keyword arguments to pass to the model and tokenizer. kwargs: Extra arguments to pass to the model and tokenizer. Returns: An object of IpexLLM. TFNrr)rr r rrrrrfrom_model_id_low_bitQszIpexLLM.from_model_id_low_bitF)r rrrrr rrc Csz$ddlm}m} ddlm} m} Wnty>tdYn0|pFi} |pNi}|pV|} d| vrhd| d<| ddvrtd| dd | d}z| j | fi| }Wn$t y| j | fi| }Yn0d | vrd d | D} d d d}|s|durd}||d<n d}||d<nd}z|j | |||| d}Wn(t yb|j ||||| d}Yn0| ||f|||| d|S)Nr) AutoModelAutoModelForCausalLM) AutoTokenizerLlamaTokenizerzCould not import ipex-llm. Please install `ipex-llm` properly following installation guides: https://github.com/intel-analytics/ipex-llm?tab=readme-ov-file#install-ipex-llm.devicecpu)r#ZxpuzXIpexLLMBgeEmbeddings currently only supports device to be 'cpu' or 'xpu', but you have: .trust_remote_codecSsi|]\}}|dkr||qS)r%r).0kvrrr sz'IpexLLM._load_model..T)Z use_cacher%from_pretrainedrrZ load_low_bit)load_function_namer load_kwargsr )r r r r )Zipex_llm.transformersrr transformersr r! ImportError ValueErrorpopr* Exceptionitems_load_model_generalto)rr rrrrr rrrr r!Z _model_kwargsZ _tokenizer_idr"r r,r+r rrrrss|            zIpexLLM._load_model) model_classr+r r,r rc Cslz$t||}||fii||WStyf}z*td|jd|d|WYd}~n d}~00dS)z,General function to attempt to load a model.zFailed to load model using r$z: N)getattrr1loggererror__name__)r5r+r r,r Z load_functionerrrr3s  zIpexLLM._load_model_general)rcCs|j|jdS)zGet the identifying parameters.r r r;selfrrr_identifying_paramsszIpexLLM._identifying_paramscCsdS)Nzipex-llmrr<rrr _llm_typeszIpexLLM._llm_type)promptstop run_managerrrc Ks@|jrddlm}|jj|dd}||jj}||jddd}|durtddlm }ddl m } || ||jg} nd} |jj |f|| d |} |jj | ddd } | S|jj|dd}||jj}|durddlm }ddl m } || ||jg} nd} |jj |fd | i|} |jj | ddd t|d} | SdS) Nr) TextStreamerpt)Zreturn_tensorsT)Z skip_promptskip_special_tokens)StoppingCriteriaList)StopSequenceCriteria)streamerstopping_criteria)rErI)rr-rCr encoder4r r"Z)transformers.generation.stopping_criteriarFZtransformers.tools.agentsrGgeneratedecodelen) r=r@rArBrrCZ input_idsrHrFrGrIoutputtextrrr_callsX        z IpexLLM._call)N)N)NFNFNN)NN) r9 __module__ __qualname____doc__DEFAULT_MODEL_IDr str__annotations__r rdictr rr rboolrZ model_config classmethodrrrr staticmethodr3propertyrr>r?rrrPrrrrr s     )!_ r )loggingtypingrrrrZlangchain_core.callbacksrZ#langchain_core.language_models.llmsrZpydanticrrT getLoggerr9r7r rrrrs