a agC@sdZddlmZddlmZmZmZmZddlZ ddl m Z m Z m Z ddlmZddlmZddlmZmZdd lmZdd lmZmZdd lmZed d dZGdddeeZGdddeZGdddeeZ GdddeeZ!dS)z@A chain for comparing the output of two models using embeddings.)Enum)AnyDictListOptionalN)AsyncCallbackManagerForChainRunCallbackManagerForChainRun Callbacks) Embeddings)pre_init) ConfigDictField)Chain)PairwiseStringEvaluatorStringEvaluatorRUN_KEYreturnc CsVzddlm}Wn>tyNzddlm}WntyHtdYn0Yn0|S)zaCreate an Embeddings object. Returns: Embeddings: The created Embeddings object. rOpenAIEmbeddingstCould not import OpenAIEmbeddings. Please install the OpenAIEmbeddings package using `pip install langchain-openai`.)langchain_openair ImportError%langchain_community.embeddings.openairrz/var/www/html/cobodadashboardai.evdpl.com/venv/lib/python3.9/site-packages/langchain/evaluation/embedding_distance/base.py_embedding_factorys  rc@s$eZdZdZdZdZdZdZdZdS)EmbeddingDistancea Embedding Distance Metric. Attributes: COSINE: Cosine distance metric. EUCLIDEAN: Euclidean distance metric. MANHATTAN: Manhattan distance metric. CHEBYSHEV: Chebyshev distance metric. HAMMING: Hamming distance metric. ZcosineZ euclideanZ manhattanZ chebyshevZhammingN) __name__ __module__ __qualname____doc__COSINE EUCLIDEAN MANHATTAN CHEBYSHEVHAMMINGrrrrr,s  rc@s:eZdZUdZeedZeed<ee j dZ e ed<e e eefe eefdddZed d Zeeed d d ZeedddZe edddZeejejejdddZeejejejdddZeejejejdddZeejejejdddZ eejejejdddZ!eje"dd d!Z#d"S)#_EmbeddingDistanceChainMixina0Shared functionality for embedding distance evaluators. Attributes: embeddings (Embeddings): The embedding objects to vectorize the outputs. distance_metric (EmbeddingDistance): The distance metric to use for comparing the embeddings. )default_factory embeddings)defaultdistance_metric)valuesrcCs|d}g}zddlm}||Wnty:Yn0zddlm}||WntyhYn0|svtdt|t|rz ddl}WntytdYn0|S)zValidate that the TikTok library is installed. Args: values (Dict[str, Any]): The values to validate. Returns: Dict[str, Any]: The validated values. r*rrrNzThe tiktoken library is required to use the default OpenAI embeddings with embedding distance evaluators. Please either manually select a different Embeddings object or install tiktoken using `pip install tiktoken`.) getrrappendrr isinstancetupletiktoken)clsr-r*Ztypes_rr2rrr_validate_tiktoken_installedJs0        z9_EmbeddingDistanceChainMixin._validate_tiktoken_installedT)Zarbitrary_types_allowedrcCsdgS)zgReturn the output keys of the chain. Returns: List[str]: The output keys. scorerselfrrr output_keys|sz(_EmbeddingDistanceChainMixin.output_keys)resultrcCs$d|di}t|vr |t|t<|S)Nr5r)r7r9parsedrrr_prepare_outputs  z,_EmbeddingDistanceChainMixin._prepare_output)metricrc CsNtj|jtj|jtj|jtj|jtj |j i}||vr<||St d|dS)zGet the metric function for the given metric name. Args: metric (EmbeddingDistance): The metric name. Returns: Any: The metric function. zInvalid metric: N) rr#_cosine_distancer$_euclidean_distancer%_manhattan_distancer&_chebyshev_distancer'_hamming_distance ValueError)r7r<Zmetricsrrr _get_metrics z(_EmbeddingDistanceChainMixin._get_metric)abrcCs:zddlm}Wnty*tdYn0d|||S)zCompute the cosine distance between two vectors. Args: a (np.ndarray): The first vector. b (np.ndarray): The second vector. Returns: np.ndarray: The cosine distance. r)cosine_similarityzThe cosine_similarity function is required to compute cosine distance. Please install the langchain-community package using `pip install langchain-community`.g?)Zlangchain_community.utils.mathrFr)rDrErFrrrr=s   z-_EmbeddingDistanceChainMixin._cosine_distancecCstj||S)zCompute the Euclidean distance between two vectors. Args: a (np.ndarray): The first vector. b (np.ndarray): The second vector. Returns: np.floating: The Euclidean distance. )npZlinalgZnormrDrErrrr>s z0_EmbeddingDistanceChainMixin._euclidean_distancecCstt||S)zCompute the Manhattan distance between two vectors. Args: a (np.ndarray): The first vector. b (np.ndarray): The second vector. Returns: np.floating: The Manhattan distance. )rGsumabsrHrrrr?s z0_EmbeddingDistanceChainMixin._manhattan_distancecCstt||S)zCompute the Chebyshev distance between two vectors. Args: a (np.ndarray): The first vector. b (np.ndarray): The second vector. Returns: np.floating: The Chebyshev distance. )rGmaxrJrHrrrr@s z0_EmbeddingDistanceChainMixin._chebyshev_distancecCst||kS)zCompute the Hamming distance between two vectors. Args: a (np.ndarray): The first vector. b (np.ndarray): The second vector. Returns: np.floating: The Hamming distance. )rGmeanrHrrrrAs z._EmbeddingDistanceChainMixin._hamming_distance)vectorsrcCs6||j}||ddd|ddd}|S)zCompute the score based on the distance metric. Args: vectors (np.ndarray): The input vectors. Returns: float: The computed score. r)rCr,Zreshapeitem)r7rMr<r5rrr_compute_scores &z+_EmbeddingDistanceChainMixin._compute_scoreN)$rr r!r"r rr*r __annotations__rr#r,r rstrrr4r Z model_configpropertyrr8dictr;rC staticmethodrGZndarrayr=Zfloatingr>r?r@rAfloatrQrrrrr(>s. "-    r(c @seZdZdZeedddZeedddZee edddZ de ee fe ee ee fd d d Zde ee fe ee ee fd d dZd d d d ddee eee e ee e ee fee edddZd d d d ddee eee e ee e ee fee edddZd S)EmbeddingDistanceEvalChaina"Use embedding distances to score semantic difference between a prediction and reference. Examples: >>> chain = EmbeddingDistanceEvalChain() >>> result = chain.evaluate_strings(prediction="Hello", reference="Hi") >>> print(result) {'score': 0.5} rcCsdS)zReturn whether the chain requires a reference. Returns: bool: True if a reference is required, False otherwise. Trr6rrrrequires_referencesz-EmbeddingDistanceEvalChain.requires_referencecCsd|jjdS)NZ embedding_ _distancer,valuer6rrrevaluation_name sz*EmbeddingDistanceEvalChain.evaluation_namecCsddgS)eReturn the input keys of the chain. Returns: List[str]: The input keys. prediction referencerr6rrr input_keyssz%EmbeddingDistanceEvalChain.input_keysNinputs run_managerrcCs0t|j|d|dg}||}d|iS)a0Compute the score for a prediction and reference. Args: inputs (Dict[str, Any]): The input data. run_manager (Optional[CallbackManagerForChainRun], optional): The callback manager. Returns: Dict[str, Any]: The computed score. r_r`r5rGarrayr*Zembed_documentsrQr7rcrdrMr5rrr_calls  z EmbeddingDistanceEvalChain._callcs:|j|d|dgIdH}t|}||}d|iS)a:Asynchronously compute the score for a prediction and reference. Args: inputs (Dict[str, Any]): The input data. run_manager (AsyncCallbackManagerForChainRun, optional): The callback manager. Returns: Dict[str, Any]: The computed score. r_r`Nr5r*Zaembed_documentsrGrfrQr7rcrdZembeddedrMr5rrr_acall-s   z!EmbeddingDistanceEvalChain._acallF)r` callbackstagsmetadatainclude_run_info)r_r`rlrmrnrokwargsrc Ks"|||d||||d}||S)aEvaluate the embedding distance between a prediction and reference. Args: prediction (str): The output string from the first model. reference (str): The reference string (required) callbacks (Callbacks, optional): The callbacks to use. **kwargs (Any): Additional keyword arguments. Returns: dict: A dictionary containing: - score: The embedding distance between the two predictions. r_r`rcrlrmrnror; r7r_r`rlrmrnrorpr9rrr_evaluate_stringsFsz,EmbeddingDistanceEvalChain._evaluate_stringsc s*|j||d||||dIdH}||S)aAsynchronously evaluate the embedding distance between a prediction and reference. Args: prediction (str): The output string from the first model. reference (str): The output string from the second model. callbacks (Callbacks, optional): The callbacks to use. **kwargs (Any): Additional keyword arguments. Returns: dict: A dictionary containing: - score: The embedding distance between the two predictions. rqrrNZacallr;rtrrr_aevaluate_stringshs z-EmbeddingDistanceEvalChain._aevaluate_strings)N)N)rr r!r"rTboolrYrSr]rrarrrrrhrrkr rUrurwrrrrrXsb        & rXc @seZdZdZeeedddZeedddZde ee fe e e ee fdd d Z de ee fe ee ee fdd d Zdddd deeee eee e ee fee edddZdddd deeee eee e ee fee edddZdS)"PairwiseEmbeddingDistanceEvalChainaUse embedding distances to score semantic difference between two predictions. Examples: >>> chain = PairwiseEmbeddingDistanceEvalChain() >>> result = chain.evaluate_string_pairs(prediction="Hello", prediction_b="Hi") >>> print(result) {'score': 0.5} rcCsddgS)r^r_ prediction_brr6rrrrasz-PairwiseEmbeddingDistanceEvalChain.input_keyscCsd|jjdS)NZpairwise_embedding_rZr[r6rrrr]sz2PairwiseEmbeddingDistanceEvalChain.evaluation_nameNrbcCs0t|j|d|dg}||}d|iS)aCompute the score for two predictions. Args: inputs (Dict[str, Any]): The input data. run_manager (CallbackManagerForChainRun, optional): The callback manager. Returns: Dict[str, Any]: The computed score. r_rzr5rergrrrrhs z(PairwiseEmbeddingDistanceEvalChain._callcs:|j|d|dgIdH}t|}||}d|iS)a/Asynchronously compute the score for two predictions. Args: inputs (Dict[str, Any]): The input data. run_manager (AsyncCallbackManagerForChainRun, optional): The callback manager. Returns: Dict[str, Any]: The computed score. r_rzNr5rirjrrrrks   z)PairwiseEmbeddingDistanceEvalChain._acallF)rlrmrnro)r_rzrlrmrnrorprc Ks"|||d||||d}||S)aEvaluate the embedding distance between two predictions. Args: prediction (str): The output string from the first model. prediction_b (str): The output string from the second model. callbacks (Callbacks, optional): The callbacks to use. tags (List[str], optional): Tags to apply to traces metadata (Dict[str, Any], optional): metadata to apply to **kwargs (Any): Additional keyword arguments. Returns: dict: A dictionary containing: - score: The embedding distance between the two predictions. r_rzrrrs r7r_rzrlrmrnrorpr9rrr_evaluate_string_pairssz9PairwiseEmbeddingDistanceEvalChain._evaluate_string_pairsc s*|j||d||||dIdH}||S)aAsynchronously evaluate the embedding distance between two predictions. Args: prediction (str): The output string from the first model. prediction_b (str): The output string from the second model. callbacks (Callbacks, optional): The callbacks to use. tags (List[str], optional): Tags to apply to traces metadata (Dict[str, Any], optional): metadata to apply to traces **kwargs (Any): Additional keyword arguments. Returns: dict: A dictionary containing: - score: The embedding distance between the two predictions. r{rrNrvr|rrr_aevaluate_string_pairss z:PairwiseEmbeddingDistanceEvalChain._aevaluate_string_pairs)N)N)rr r!r"rTrrSrar]rrrrrhrrkr rxrUr}r~rrrrrysZ       ( ry)"r"enumrtypingrrrrnumpyrGZ langchain_core.callbacks.managerrrr Zlangchain_core.embeddingsr Zlangchain_core.utilsr Zpydanticr r Zlangchain.chains.baserZlangchain.evaluation.schemarrZlangchain.schemarrrSrr(rXryrrrrs&     :