a bg]@sxddlZddlmZmZmZmZmZmZddlZ ddl m Z ddl m Z ddlmZddlmZdZGdd d eZdS) N)AnyDictIterableListOptionalTuple)Document) Embeddings) VectorStore)maximal_marginal_relevancec @seZdZdZeeddddZddddZee edd d Z de e e ee edddZd?e e e ee edddZefdde eee ee e fee eeeefdddZefdde ee ee e fee eeefdddZefdde ee ee e fee eeefdddZefdde ee ee e fee eeefdddZefdde ee ee e fee eeefdd d!Zefdde eee ee e fee edd"d#Zefdde eee ee e fee edd$d%Z efdde ee ee e fee edd&d'Z!efdde ee ee e fee edd(d)Z"ed*d+fdde eeeee ee e fee ed,d-d.Z#ed*d+fdde eeeee ee e fee ed,d/d0Z$d@dde eeee ee e fee ed2d3d4Z%ed*d+fdde eeee ee e fee ed2d5d6Z&e'dAe e ee e eedd7d8d9Z(e'dBe e ee e eedd7d:d;Z)dS)CSurrealDBStorea/ SurrealDB as Vector Store. To use, you should have the ``surrealdb`` python package installed. Args: embedding_function: Embedding function to use. dburl: SurrealDB connection url ns: surrealdb namespace for the vector store. (default: "langchain") db: surrealdb database for the vector store. (default: "database") collection: surrealdb collection for the vector store. (default: "documents") (optional) db_user and db_pass: surrealdb credentials Example: .. code-block:: python from langchain_community.vectorstores.surrealdb import SurrealDBStore from langchain_community.embeddings import HuggingFaceEmbeddings model_name = "sentence-transformers/all-mpnet-base-v2" embedding_function = HuggingFaceEmbeddings(model_name=model_name) dburl = "ws://localhost:8000/rpc" ns = "langchain" db = "docstore" collection = "documents" db_user = "root" db_pass = "root" sdb = SurrealDBStore.from_texts( texts=texts, embedding=embedding_function, dburl, ns, db, collection, db_user=db_user, db_pass=db_pass) N)embedding_functionkwargsreturnc Kszddlm}Wn.ty>}ztd|WYd}~n d}~00|dd|_|jdddkrn||j|_ntd|d d |_|d d |_|d d|_ ||_ ||_ dS)Nr)SurrealzZCannot import from surrealdb. please install with `pip install surrealdb`.dburlzws://localhost:8000/rpcwsz6Only websocket connections are supported at this time.nsZ langchaindbZdatabase collectionZ documents) Z surrealdbr ImportErrorpoprsdb ValueErrorrrrrr)selfrrrerx/var/www/html/cobodadashboardai.evdpl.com/venv/lib/python3.9/site-packages/langchain_community/vectorstores/surrealdb.py__init__5s"zSurrealDBStore.__init__rcsp|jIdHd|jvrTd|jvrT|jd}|jd}|j||dIdH|j|j|jIdHdS)zr Initialize connection to surrealdb database and authenticate if credentials are provided NZdb_userZdb_pass)userpass)rconnectrgetZsigninZuserr)rr"passwordrrr initializeOs   zSurrealDBStore.initializecCst|jtr|jSdSN) isinstancerr rrrr embeddings[s zSurrealDBStore.embeddingstexts metadatasrrc s|jt|}g}t|D]h\}}|||d}|durT|t|krT|||d<ng|d<|j|j|IdH} || ddq|S)zAdd list of text along with embeddings to the vector store asynchronously Args: texts (Iterable[str]): collection of text to add to the database Returns: List of ids for the newly inserted documents )text embeddingNmetadatarid) rZembed_documentslist enumeratelenrcreaterappend) rr-r.rr+idsidxr/datarecordrrr aadd_textscs zSurrealDBStore.aadd_textsc sBdttttttttdfdd }t|||fi|S)zAdd list of text along with embeddings to the vector store Args: texts (Iterable[str]): collection of text to add to the database Returns: List of ids for the newly inserted documents Nr,cs(IdHj||fi|IdHSr(r'r<)r-r.rr*rr _add_textssz,SurrealDBStore.add_texts.._add_texts)N)rstrrrdictrasynciorun)rr-r.rr>rr*r add_textss zSurrealDBStore.add_textsr8rrcsv|dur jjIdHdSt|tr@j|IdHdSt|trrt|dkrrfdd|DIdH}dSdS)aDelete by document ID asynchronously. Args: ids: List of ids to delete. **kwargs: Other keyword arguments that subclasses might use. Returns: Optional[bool]: True if deletion is successful, False otherwise. NTrcsg|]}j|IdHqSr()rdelete).0r2r*rr z*SurrealDBStore.adelete..F)rrErr)r?r3r5)rr8r_rr*radeletes zSurrealDBStore.adeletec s8ttttttdfdd }t||fi|S)a Delete by document ID. Args: ids: List of ids to delete. **kwargs: Other keyword arguments that subclasses might use. Returns: Optional[bool]: True if deletion is successful, False otherwise. rDcs(IdHjfd|i|IdHS)Nr8)r'rJ)r8rr*rr_deletesz&SurrealDBStore.delete.._delete)rrr?rboolrArB)rr8rrKrr*rrEs"zSurrealDBStore.delete)filter)r0krMrrcs|j|||ddd}d}|rp|D]J}t||ttfvrNd||d}n ||}|d|d|d7}q$d |d d |d } |j| |Id H} t| dkrgS| d} | ddkrddlm } | dd} | | dd| dDS)aRun similarity search for query embedding asynchronously and return documents and scores Args: embedding (List[float]): Query embedding. k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar along with scores score_thresholdr)rr0rNrO'z and metadata.z =  u select id, text, metadata, embedding, vector::similarity::cosine(embedding, $embedding) as similarity from ⟨rub⟩ where vector::similarity::cosine(embedding, $embedding) >= $score_threshold z4 order by similarity desc LIMIT $k; NstatusOK)SurrealExceptionresultz Unknown ErrorcSs@g|]8}t|dd|di|dp&id|d|dfqS)r/r2r1)Z page_contentr1 similarityr0)rr%)rFdocrrrrG s zKSurrealDBStore._asimilarity_search_by_vector_with_score..) rr%typer?rLrqueryr5Z surrealdb.wsrU)rr0rNrMrargsZ custom_filterkeyZ filter_valuerZresultsrVrUerrrrr(_asimilarity_search_by_vector_with_scores8         z7SurrealDBStore._asimilarity_search_by_vector_with_score)rZrNrMrrcs4|j|}dd|j||fd|i|IdHDS)afRun similarity search asynchronously and return relevance scores Args: query (str): Query k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar along with relevance scores cSsg|]\}}}||fqSrrrFdocumentrWrIrrrrG*szKSurrealDBStore.asimilarity_search_with_relevance_scores..rMNr embed_queryr_rrZrNrMrZquery_embeddingrrr(asimilarity_search_with_relevance_scoress  z7SurrealDBStore.asimilarity_search_with_relevance_scoresc s2ttttfdfdd }t|S)aeRun similarity search synchronously and return relevance scores Args: query (str): Query k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar along with relevance scores r!cs,IdHjfdiIdHSNrM)r'rerrMrNrrZrrr(_similarity_search_with_relevance_scoresFszhSurrealDBStore.similarity_search_with_relevance_scores.._similarity_search_with_relevance_scoresrrrfloatrArB)rrZrNrMrrhrrgr'similarity_search_with_relevance_scores3sz6SurrealDBStore.similarity_search_with_relevance_scorescs4|j|}dd|j||fd|i|IdHDS)anRun similarity search asynchronously and return distance scores Args: query (str): Query k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar along with relevance distance scores cSsg|]\}}}||fqSrrr`rrrrGcsz@SurrealDBStore.asimilarity_search_with_score..rMNrbrdrrrasimilarity_search_with_scorePs  z,SurrealDBStore.asimilarity_search_with_scorec s2ttttfdfdd }t|S)amRun similarity search synchronously and return distance scores Args: query (str): Query k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar along with relevance distance scores r!cs,IdHjfdiIdHSrf)r'rlrrgrr_similarity_search_with_scoreszRSurrealDBStore.similarity_search_with_score.._similarity_search_with_scoreri)rrZrNrMrrmrrgrsimilarity_search_with_scorels&z+SurrealDBStore.similarity_search_with_scorecs(dd|j||fd|i|IdHDS)adRun similarity search on query embedding asynchronously Args: embedding (List[float]): Query embedding k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar to the query cSsg|]\}}}|qSrr)rFrarIrrrrGsz?SurrealDBStore.asimilarity_search_by_vector..rMN)r_)rr0rNrMrrrrasimilarity_search_by_vectors z+SurrealDBStore.asimilarity_search_by_vectorc s*ttdfdd }t|S)aURun similarity search on query embedding Args: embedding (List[float]): Query embedding k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar to the query r!cs,IdHjfdiIdHSrf)r'rorr0rMrNrrrr_similarity_search_by_vectorszPSurrealDBStore.similarity_search_by_vector.._similarity_search_by_vectorrrrArB)rr0rNrMrrqrrprsimilarity_search_by_vectorsz*SurrealDBStore.similarity_search_by_vectorcs*|j|}|j||fd|i|IdHS)aDRun similarity search on query asynchronously Args: query (str): Query k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar to the query rMN)rrcrordrrrasimilarity_searchs z!SurrealDBStore.asimilarity_searchc s*ttdfdd }t|S)a5Run similarity search on query Args: query (str): Query k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar to the query r!cs,IdHjfdiIdHSrf)r'rtrrgrr_similarity_searchsz._similarity_searchrr)rrZrNrMrrurrgrsimilarity_searchsz SurrealDBStore.similarity_search?)r0rNfetch_k lambda_multrMrrc sh|j||fd|i|IdH}dd|Ddd|D}ttj|tjd|||d} fdd| DS) aHReturn docs selected using the maximal marginal relevance. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents. Args: embedding: Embedding to look up documents similar to. k: Number of Documents to return. Defaults to 4. fetch_k: Number of Documents to fetch to pass to MMR algorithm. lambda_mult: Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents selected by maximal marginal relevance. rMNcSsg|] }|dqS)rrrFsubrrrrG rHzKSurrealDBStore.amax_marginal_relevance_search_by_vector..cSsg|] }|dqS)rr{rrrrGrH)Zdtype)rNrzcsg|] }|qSrr)rFidocsrrrGrH)r_r nparrayZfloat32) rr0rNryrzrMrrVr+Z mmr_selectedrrr(amax_marginal_relevance_search_by_vectors  z7SurrealDBStore.amax_marginal_relevance_search_by_vectorc s.ttdfdd }t|S)aIReturn docs selected using the maximal marginal relevance. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents. Args: embedding: Embedding to look up documents similar to. k: Number of Documents to return. Defaults to 4. fetch_k: Number of Documents to fetch to pass to MMR algorithm. lambda_mult: Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents selected by maximal marginal relevance. r!cs0IdHjfdiIdHSrf)r'rrr0ryrMrNrrzrrr(_max_marginal_relevance_search_by_vector6szhSurrealDBStore.max_marginal_relevance_search_by_vector.._max_marginal_relevance_search_by_vectorrr)rr0rNryrzrMrrrrr'max_marginal_relevance_search_by_vectors"z6SurrealDBStore.max_marginal_relevance_search_by_vectorr )rZrNryrzrMrrc s2|j|}|j||||fd|i|IdH}|S)a@Return docs selected using the maximal marginal relevance. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents. Args: query: Text to look up documents similar to. k: Number of Documents to return. Defaults to 4. fetch_k: Number of Documents to fetch to pass to MMR algorithm. lambda_mult: Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents selected by maximal marginal relevance. rMN)rrcr) rrZrNryrzrMrr0rrrramax_marginal_relevance_search>s  z-SurrealDBStore.amax_marginal_relevance_searchc s.ttdfdd }t|S)a?Return docs selected using the maximal marginal relevance. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents. Args: query: Text to look up documents similar to. k: Number of Documents to return. Defaults to 4. fetch_k: Number of Documents to fetch to pass to MMR algorithm. lambda_mult: Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents selected by maximal marginal relevance. r!cs0IdHjfdiIdHSrf)r'rrryrMrNrrzrZrrr_max_marginal_relevance_search}szTSurrealDBStore.max_marginal_relevance_search.._max_marginal_relevance_searchrr)rrZrNryrzrMrrrrrmax_marginal_relevance_searchas"z,SurrealDBStore.max_marginal_relevance_search)r-r0r.rrcs<||fi|}|IdH|j||fi|IdH|S)aCreate SurrealDBStore from list of text asynchronously Args: texts (List[str]): list of text to vectorize and store embedding (Optional[Embeddings]): Embedding function. dburl (str): SurrealDB connection url (default: "ws://localhost:8000/rpc") ns (str): surrealdb namespace for the vector store. (default: "langchain") db (str): surrealdb database for the vector store. (default: "database") collection (str): surrealdb collection for the vector store. (default: "documents") (optional) db_user and db_pass: surrealdb credentials Returns: SurrealDBStore object initialized and ready for use.Nr=clsr-r0r.rrrrr afrom_textsszSurrealDBStore.afrom_textscKs t|j|||fi|}|S)aCreate SurrealDBStore from list of text Args: texts (List[str]): list of text to vectorize and store embedding (Optional[Embeddings]): Embedding function. dburl (str): SurrealDB connection url ns (str): surrealdb namespace for the vector store. (default: "langchain") db (str): surrealdb database for the vector store. (default: "database") collection (str): surrealdb collection for the vector store. (default: "documents") (optional) db_user and db_pass: surrealdb credentials Returns: SurrealDBStore object initialized and ready for use.)rArBrrrrr from_textsszSurrealDBStore.from_texts)N)N)N)N)r rwrx)N)N)*__name__ __module__ __qualname____doc__r rr r'propertyrr+rr?rr@r<rCrLrJrE DEFAULT_Krjintrrrr_rerkrlrnrorsrtrvrrrr classmethodrrrrrrr s(            N         1 ( & $  r )rAtypingrrrrrrnumpyrZlangchain_core.documentsrZlangchain_core.embeddingsr Zlangchain_core.vectorstoresr Z&langchain_community.vectorstores.utilsr rr rrrrs