a bg@sddlmZddlmZmZmZmZmZddlZ ddl m Z ddl m Z ddlmZddlmZddlmZdd lmZdd lmZGd d d eeZGd ddeZdS))Enum)AnyDictListOptionalUnionN)CallbackManagerForRetrieverRun)Document) Embeddings) BaseRetriever) get_fields) ConfigDict)maximal_marginal_relevancec@seZdZdZdZdZdS) SearchTypez-Enumerator of the types of search to perform. similaritymmrN)__name__ __module__ __qualname____doc__rrrru/var/www/html/cobodadashboardai.evdpl.com/venv/lib/python3.9/site-packages/langchain_community/retrievers/docarray.pyrsrc@seZdZUdZdZeed<eed<eed<eed<e j Z e ed<dZ e ed <dZeeed <ed d Zeeeed ddZeje eeeeefefdddZejeedddZejeedddZeeeefefedddZdS)DocArrayRetrievera`DocArray Document Indices` retriever. Currently, it supports 5 backends: InMemoryExactNNIndex, HnswDocumentIndex, QdrantDocumentIndex, ElasticDocIndex, and WeaviateDocumentIndex. Args: index: One of the above-mentioned index instances embeddings: Embedding model to represent text as vectors search_field: Field to consider for searching in the documents. Should be an embedding/vector/tensor. content_field: Field that represents the main content in your document schema. Will be used as a `page_content`. Everything else will go into `metadata`. search_type: Type of search to perform (similarity / mmr) filters: Filters applied for document retrieval. top_k: Number of documents to return Nindex embeddings search_field content_field search_typetop_kfiltersT)Zarbitrary_types_allowed)query run_managerreturncCsXt|j|}|jtjkr*||}n*|jtjkrB| |}nt d|jd|S)zGet documents relevant for a query. Args: query: string to find relevant documents for Returns: List of relevant documents z Search type z5 does not exist. Choose either 'similarity' or 'mmr'.) nparrayrZ embed_queryrrr_similarity_searchr _mmr_search ValueError)selfr!r" query_embresultsrrr_get_relevant_documents5s     z)DocArrayRetriever._get_relevant_documents)r*rr#c Csddlm}m}i}|j}t|j|r6|j|d<d}n"t|j|rN|j|d<n |j|d<|jr|jj||dj fi|j |d}|j |}t |d r|j }|d |}n|jj|||d j }|S) a Perform a search using the query embedding and return top_k documents. Args: query_emb: Query represented as an embedding top_k: Number of documents to return Returns: A list of top_k documents matching the query r)ElasticDocIndexWeaviateDocumentIndexZ where_filterr!Z filter_query)r!r)limit documentsN)r!rr0)Zdocarray.indexr-r.r isinstancerr Z build_queryfindfilterbuildZ execute_queryhasattrr1) r)r*rr-r.Z filter_argsrr!docsrrr_searchQs4         zDocArrayRetriever._search)r*r#cs&j|jd}fdd|D}|S)z Perform a similarity search. Args: query_emb: Query represented as an embedding Returns: A list of documents most similar to the query r*rcsg|]}|qSr_docarray_to_langchain_doc.0docr)rr z8DocArrayRetriever._similarity_search..)r8r)r)r*r7r+rr?rr&s z$DocArrayRetriever._similarity_searchcsDj|ddt|fddDjd}fdd|D}|S)z Perform a maximal marginal relevance (mmr) search. Args: query_emb: Query represented as an embedding Returns: A list of diverse documents related to the query r9cs,g|]$}t|tr|jn t|jqSr)r2dictrgetattrr<r?rrr@s z1DocArrayRetriever._mmr_search..)kcsg|]}|qSrr:)r=idxr7r)rrr@rA)r8rr)r)r*Z mmr_selectedr+rrGrr's   zDocArrayRetriever._mmr_search)r>r#cCst|tr|nt|}|j|vr6td|jdtt|trL||jn t||jd}|D]F}t|trv||nt||}t|tt t t fr`||jkr`||j |<q`|S)a; Convert a DocArray document (which also might be a dict) to a langchain document format. DocArray document can contain arbitrary fields, so the mapping is done in the following way: page_content <-> content_field metadata <-> all other fields excluding tensors and embeddings (so float, int, string) Args: doc: DocArray document Returns: Document in langchain format Raises: ValueError: If the document doesn't contain the content field z.Document does not contain the content field - .)Z page_content) r2rCkeysr rr(r rDstrintfloatboolmetadata)r)r>fieldsZlc_docnamevaluerrrr;s$     z,DocArrayRetriever._docarray_to_langchain_doc)rrrrrr__annotations__r rJrrrrrKr rr Z model_configrrr r,r$Zndarrayrrr8r&r'r;rrrrrs*     .r)enumrtypingrrrrrnumpyr$Zlangchain_core.callbacksrZlangchain_core.documentsr Zlangchain_core.embeddingsr Zlangchain_core.retrieversr Zlangchain_core.utils.pydanticr Zpydanticr Z&langchain_community.vectorstores.utilsrrJrrrrrrs