a bgR@sddlmZddlZddlZddlZddlmZmZmZm Z m Z m Z m Z m Z mZddlZddlmZddlmZddlmZddlmZmZerddlmZdd lmZmZee Z!d d d d Z"e dddZ#GdddeZ$dS)) annotationsN) TYPE_CHECKINGAnyCallableIterableListOptionalTupleTypeVarUnion)Document) Embeddings) VectorStore)DistanceStrategymaximal_marginal_relevanceClient)NeighborVectorDistanceMetricrreturnc CsDzddlm}Wn.ty>}ztd|WYd}~n d}~00|S)NrrzoCould not import aerospike_vector_search python package. Please install it with `pip install aerospike_vector`.)aerospike_vector_searchr ImportError)rerx/var/www/html/cobodadashboardai.evdpl.com/venv/lib/python3.9/site-packages/langchain_community/vectorstores/aerospike.py_import_aerospike#srAVST Aerospike)boundc@seZdZdZdddddejfdddd dddd d d d d ZeddddZdddddZ dddddZ e dddddZ dQdd"d#d d$d d%d&d'd( d)d*Z dRd#d d&d+d,d-d.ZdSdd$d#d d&d0d1d2d3ZdTdd$d#d d&d0d4d5d6ZdUdd$d#d d&d7d4d8d9ZdVdd$d#d d&d7d1d:d;ZdZe d?d?d@dAdBZdWdd$d$d?d#d d&d7dEdFdGZdXdd$d$d?d#d d&d7dHdIdJZedYd'dLd"ddd d#d$dMd&ddN dOdPZdS)Zrzu`Aerospike` vector store. To use, you should have the ``aerospike_vector_search`` python package installed. NZ_vector_textZ_idrzUnion[Embeddings, Callable]strz Optional[str]z7Optional[Union[DistanceStrategy, VectorDistanceMetric]]) client embedding namespace index_name vector_keytext_keyid_keyset_namedistance_strategyc Csvt} t|tstdt|| s6tdt|||_||_||_ ||_ ||_ ||_ ||_ ||_|| |_dS)aInitialize with Aerospike client. Args: client: Aerospike client. embedding: Embeddings object or Callable (deprecated) to embed text. namespace: Namespace to use for storing vectors. This should match index_name: Name of the index previously created in Aerospike. This vector_key: Key to use for vector in metadata. This should match the key used during index creation. text_key: Key to use for text in metadata. id_key: Key to use for id in metadata. set_name: Default set name to use for storing vectors. distance_strategy: Distance strategy to use for similarity search This should match the distance strategy used during index creation. z`Passing in `embedding` as a Callable is deprecated. Please pass in an Embeddings object instead.zDclient should be an instance of aerospike_vector_search.Client, got N)r isinstancer warningswarn ValueErrortype_client _embedding _text_key _vector_key_id_key _index_name _namespace _set_nameconvert_distance_strategy_distance_strategy) selfr"r#r$r%r&r'r(r)r* aerospikerrr__init__7s(  zAerospike.__init__zOptional[Embeddings]rcCst|jtr|jSdS)z/Access the query embedding object if available.N)r+r1r r:rrr embeddingsms zAerospike.embeddingsz Iterable[str]zList[List[float]])textsrcs.tjtrjt|Sfdd|DS)zEmbed search docs.csg|]}|qSr)r1).0tr=rr xz.Aerospike._embed_documents..)r+r1r Zembed_documentslist)r:r?rr=r_embed_documentsts zAerospike._embed_documentsz List[float])textrcCs"t|jtr|j|S||S)zEmbed query text.)r+r1r Z embed_query)r:rFrrr _embed_queryzs  zAerospike._embed_queryz-Union[VectorDistanceMetric, DistanceStrategy]r)r*rcCsVddlm}t|tr|S||jkr*tjS||jkr:tjS||jkrJtjStddS)z Convert Aerospikes distance strategy to langchains DistanceStrategy enum. This is a convenience method to allow users to pass in the same distance metric used to create the index. r)rDUnknown distance strategy, must be cosine, dot_product, or euclideanN) aerospike_vector_search.typesrr+rCOSINE DOT_PRODUCTZSQUARED_EUCLIDEANEUCLIDEAN_DISTANCEr.)r*rrrrr8s     z#Aerospike.convert_distance_strategyTzOptional[List[dict]]zOptional[List[str]]intboolrz List[str]) r? metadatasidsr)embedding_chunk_sizer%wait_for_indexkwargsrc KsD|dur|j}|dur|j}|r0|dur0tdt|}|pHdd|D}|r^dd|D}n|pndd|D}tdt||D]} || | |} || | |} || | |} || } t| | | D]\}}}|||j<|||j <qt| | D]0\}}|||j <|j j f|j |||d|qq|r@|j j|j |d |S) aRun more texts through the embeddings and add to the vectorstore. Args: texts: Iterable of strings to add to the vectorstore. metadatas: Optional list of metadata associated with the texts. ids: Optional list of ids to associate with the texts. set_name: Optional aerospike set name to add the texts to. batch_size: Batch size to use when adding the texts to the vectorstore. embedding_chunk_size: Chunk size to use when embedding the texts. index_name: Optional aerospike index name used for waiting for index completion. If not provided, the default index_name will be used. wait_for_index: If True, wait for the all the texts to be indexed before returning. Requires index_name to be provided. Defaults to True. kwargs: Additional keyword arguments to pass to the client upsert call. Returns: List of ids from adding the texts into the vectorstore. Nz6if wait_for_index is True, index_name must be providedcSsg|]}ttqSr)r!uuiduuid4r@_rrrrBrCz'Aerospike.add_texts..cSsg|] }|qSr)copy)r@mrrrrBrCcSsg|]}iqSrrrWrrrrBrCr)r$keyr)Z record_data)r$name)r7r5r.rDrangelenrEzipr3r2r4r0Zupsertr6Zwait_for_index_completion)r:r?rPrQr)rRr%rSrTiZ chunk_textsZ chunk_idsZchunk_metadatasr>metadatar#rFidrrr add_textssJ       zAerospike.add_textszOptional[bool])rQr)rTrc KsVddlm}|rR|D]<}z |jjf|j||d|Wq|yNYdS0qdS)a7Delete by vector ID or other criteria. Args: ids: List of ids to delete. **kwargs: Other keyword arguments to pass to client delete call. Returns: Optional[bool]: True if deletion is successful, False otherwise, None if not implemented. r)AVSServerError)r$r[r)FT)rrdr0deleter6)r:rQr)rTrdrbrrrres    zAerospike.deletezList[Tuple[Document, float]])queryk metadata_keysr%rTrcKs |j||f|||d|S)aReturn aerospike documents most similar to query, along with scores. Args: query: Text to look up documents similar to. k: Number of Documents to return. Defaults to 4. metadata_keys: List of metadata keys to return with the documents. If None, all metadata keys will be returned. Defaults to None. index_name: Name of the index to search. Overrides the default index_name. kwargs: Additional keyword arguments to pass to the search method. Returns: List of Documents most similar to the query and associated scores. rhrir%)&similarity_search_by_vector_with_scorerG)r:rgrhrir%rTrrrsimilarity_search_with_score sz&Aerospike.similarity_search_with_score)r#rhrir%rTrc Ksg}|r|j|vr|jg|}|dur,|j}|dur.rj)rk)r:r#rhrir%rTrrrsimilarity_search_by_vectorfsz%Aerospike.similarity_search_by_vectorcKs(|j|f|||d|}dd|DS)a*Return aerospike documents most similar to query. Args: query: Text to look up documents similar to. k: Number of Documents to return. Defaults to 4. metadata_keys: List of metadata keys to return with the documents. If None, all metadata keys will be returned. Defaults to None. index_name: Optional name of the index to search. Overrides the default index_name. Returns: List of Documents most similar to the query and score for each rjcSsg|] \}}|qSrrrxrrrrBrCz/Aerospike.similarity_search..)rl)r:rgrhrir%rTZdocs_and_scoresrrrsimilarity_searchszAerospike.similarity_searchzCallable[[float], float]cCsB|jtjkr|jS|jtjkr$|jS|jtjkr6|jStddS)a The 'correct' relevance function may differ depending on a few things, including: - the distance / similarity metric used by the VectorStore - the scale of your embeddings (OpenAI's are unit normed. Many others are not!) - embedding dimensionality - etc. 0 is dissimilar, 1 is similar. Aerospike's relevance_fn assume euclidean and dot product embeddings are normalized to unit norm. rHN) r9rrJ_cosine_relevance_score_fnrKZ%_max_inner_product_relevance_score_fnrLZ_euclidean_relevance_score_fnr.r=rrr_select_relevance_score_fns   z$Aerospike._select_relevance_score_fnfloat)rwrcCs d|dS)zgAerospike returns cosine distance scores between [0,2] 0 is dissimilar, 1 is similar. r)rwrrrr|sz$Aerospike._cosine_relevance_score_fn?)r#rhfetch_k lambda_multrir%rTrc  s|rj|vrjg|}j|f|||d|ttj|gtjdfddD||d}|rj|vr|D]} | jjqrfdd|DS)aReturn docs selected using the maximal marginal relevance. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents. Args: embedding: Embedding to look up documents similar to. k: Number of Documents to return. Defaults to 4. fetch_k: Number of Documents to fetch to pass to MMR algorithm. lambda_mult: Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5. metadata_keys: List of metadata keys to return with the documents. If None, all metadata keys will be returned. Defaults to None. index_name: Optional name of the index to search. Overrides the default index_name. Returns: List of Documents selected by maximal marginal relevance. rj)Zdtypecsg|]}|jjqSr)rar3)r@ryr=rrrBrCzEAerospike.max_marginal_relevance_search_by_vector..)rhrcsg|] }|qSrr)r@r`)rtrrrBrC)r3rzrnparrayZfloat32rarp) r:r#rhrrrir%rTZ mmr_selectedr`r)rtr:r'max_marginal_relevance_search_by_vectors* z1Aerospike.max_marginal_relevance_search_by_vector)rgrhrrrir%rTrc Ks(||}|j||||f||d|S)aReturn docs selected using the maximal marginal relevance. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents. Args: query: Text to look up documents similar to. k: Number of Documents to return. Defaults to 4. fetch_k: Number of Documents to fetch to pass to MMR algorithm. lambda_mult: Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5. index_name: Name of the index to search. Returns: List of Documents selected by maximal marginal relevance. )rir%)rGr) r:rgrhrrrir%rTr#rrrmax_marginal_relevance_searchs z'Aerospike.max_marginal_relevance_searchtestr zOptional[dict]) r?r#rPr"r$r%rQembeddings_chunk_size client_kwargsrTrc Ks8||||fi| } | j|f||||d| p.i| S)a This is a user friendly interface that: 1. Embeds text. 2. Converts the texts into documents. 3. Adds the documents to a provided Aerospike index This is intended to be a quick way to get started. Example: .. code-block:: python from langchain_community.vectorstores import Aerospike from langchain_openai import OpenAIEmbeddings from aerospike_vector_search import Client, HostPort client = Client(seeds=HostPort(host="localhost", port=5000)) aerospike = Aerospike.from_texts( ["foo", "bar", "baz"], embedder, client, "namespace", index_name="index", vector_key="vector", distance_strategy=MODEL_DISTANCE_CALC, ) )rPrQr%rR)rc) clsr?r#rPr"r$r%rQrrrTr;rrr from_textss$(zAerospike.from_texts)NNNrMNT)NN)rfNN)rfNN)rfNN)rfNN)rfrrNN)rfrrNN)NNrNNrMN)__name__ __module__ __qualname____doc__rrLr<propertyr>rErG staticmethodr8rcrerlrkrzr{r}r|rr classmethodrrrrrr1s  6 R#"?% 8&)% __future__rloggingrUr,typingrrrrrrr r r numpyrZlangchain_core.documentsr Zlangchain_core.embeddingsr Zlangchain_core.vectorstoresrZ&langchain_community.vectorstores.utilsrrrrrIrr getLoggerrrrrrrrrrrs ,