a bg7@sdZddlmZddlZddlZddlmZmZmZm Z m Z m Z ddl m Z ddlmZddlmZertddlmZeZGd d d eZdS) z'Wrapper around Epsilla vector database.) annotationsN) TYPE_CHECKINGAnyIterableListOptionalTypeDocument) Embeddings) VectorStore)vectordbc@sDeZdZUdZdZded<dZded<dZded<eefd d d d d d dZe ddddZ dddddZ d:dddddZ d;ddddddZ ddd.dd d/d0d1d2Zeddeeed%fd3d)d d'd d d d d(d dd4 d5d6Zedeeed%fd3d/d d d d d d(d dd7 d8d9ZdS)?Epsillaar Wrapper around Epsilla vector database. As a prerequisite, you need to install ``pyepsilla`` package and have a running Epsilla vector database (for example, through our docker image) See the following documentation for how to run an Epsilla vector database: https://epsilla-inc.gitbook.io/epsilladb/quick-start Args: client (Any): Epsilla client to connect to. embeddings (Embeddings): Function used to embed the texts. db_path (Optional[str]): The path where the database will be persisted. Defaults to "/tmp/langchain-epsilla". db_name (Optional[str]): Give a name to the loaded database. Defaults to "langchain_store". Example: .. code-block:: python from langchain_community.vectorstores import Epsilla from pyepsilla import vectordb client = vectordb.Client() embeddings = OpenAIEmbeddings() db_path = "/tmp/vectorstore" db_name = "langchain_store" epsilla = Epsilla(client, embeddings, db_path, db_name) Zlangchain_storestr_LANGCHAIN_DEFAULT_DB_NAMEz/tmp/langchain-epsilla_LANGCHAIN_DEFAULT_DB_PATHZlangchain_collection_LANGCHAIN_DEFAULT_TABLE_NAMErr z Optional[str])client embeddingsdb_pathdb_namec Csz ddl}Wn.ty:}ztd|WYd}~n d}~00t||jj|jjjfsftdt |||_ ||_ ||_ t j|_|j j||d|j j|ddS)z%Initialize with necessary components.rNziCould not import pyepsilla python package. Please install pyepsilla package with `pip install pyepsilla`.zbclient should be an instance of pyepsilla.vectordb.Client or pyepsilla.cloud.client.Vectordb, got )rrr) pyepsilla ImportError isinstancer ZClientcloudrZVectordb TypeErrortype_clientZ_db_name _embeddingsrr_collection_nameZload_dbZuse_db)selfrrrrrer#v/var/www/html/cobodadashboardai.evdpl.com/venv/lib/python3.9/site-packages/langchain_community/vectorstores/epsilla.py__init__4s, zEpsilla.__init__zOptional[Embeddings])returncCs|jS)N)rr!r#r#r$rSszEpsilla.embeddingsNone)collection_namer&cCs ||_dS)z~ Set default collection to use. Args: collection_name (str): The name of the collection. N)r r!r)r#r#r$use_collectionWszEpsilla.use_collectioncCs|s |j}|j|dS)z Clear data in a collection. Args: collection_name (Optional[str]): The name of the collection. If not provided, the default collection will be used. N)r rZ drop_tabler*r#r#r$ clear_data`szEpsilla.clear_dataNzOptional[List[str]]z List[dict])r)response_fieldsr&cCsT|s |j}|jj||d\}}|dkrLtd|dtd|d|dS)aGet the collection. Args: collection_name (Optional[str]): The name of the collection to retrieve data from. If not provided, the default collection will be used. response_fields (Optional[List[str]]): List of field names in the result. If not specified, all available fields will be responded. Returns: A list of the retrieved data. ) table_namer.zFailed to get records: message Error: {}.result)r rgetloggererror Exceptionformat)r!r)r. status_coderesponser#r#r$r4ls z Epsilla.getlistzOptional[list[dict]])r/r metadatasr&c CsH|s tdt|d}dddddddd |d g}|durd d |D}|D]}|D]|\}} ||vrnq\t| tr~d} n@t| trd} n0t| trd } n t| trd} ntd|d||| d||q\qP|j j ||d\} } | dkrD| dkrt d|dn,t d|d| dtd| ddS)NzEmbeddings list is empty.ridINT)namedataTypetextSTRINGrZ VECTOR_FLOAT)r?r@ dimensionscSsg|] }|dqS)r?r#).0fieldr#r#r$ z.Epsilla._create_collection..FLOATZBOOLzUnsupported data type for .)Z table_fieldsr0iz#Continuing with the existing table zFailed to create collection : r1r2) ValueErrorlenitemsrrintfloatboolappendrZ create_tabler5infor6r7r8) r!r/rr<dimfields field_namesmetadatakeyvalueZd_typer9r:r#r#r$_create_collectionsD         zEpsilla._create_collectionFz Iterable[str]zOptional[List[dict]]zOptional[bool]z List[str])textsr<r)drop_oldkwargsr&c sP|s j}n|_|r$jj|dt|}zj|}Wn$ty`fdd|D}Yn0t|dkr|t dgSj |||ddd|D}g}t |D]P\} } | || || d} |d ur||  } | D]\} }|| | <q| | qjj||d \}}|d krBtd |d |dtd|ddd|DS)a Embed texts and add them to the database. Args: texts (Iterable[str]): The texts to embed. metadatas (Optional[List[dict]]): Metadata dicts attached to each of the texts. Defaults to None. collection_name (Optional[str]): Which collection to use. Defaults to "langchain_collection". If provided, default collection name will be set as well. drop_old (Optional[bool]): Whether to drop the previous collection and create a new one. Defaults to False. Returns: List of ids of the added texts. rcsg|]}j|qSr#)r embed_query)rDxr'r#r$rFrGz%Epsilla.add_texts..rzNothing to insert, skipping.)r/rr<cSsg|]}ttqSr#)hashuuiduuid4)rD_r#r#r$rFrGr=rArN)r/recordsr0zFailed to add records to rJr1r2cSsg|] }t|qSr#)r)rDr=r#r#r$rFrG)r rZdrop_dbr;rZembed_documentsNotImplementedErrorrLr5debugrY enumeraterMrQinsertr6r7r8)r!rZr<r)r[r\rZidsrdindexr=recordrVrWrXr9r:r#r'r$ add_textssJ         zEpsilla.add_textsrNzList[Document])querykr)r\r&c s|s |j}|j|}|jj|d||d\}}|dkr^td|ddtd|dgdt t fd d |d S) a Return the documents that are semantically most relevant to the query. Args: query (str): String to query the vectorstore with. k (Optional[int]): Number of documents to return. Defaults to 4. collection_name (Optional[str]): Collection to use. Defaults to "langchain_store" or the one provided before. Returns: List of documents that are semantically most relevant to the query r)r/Z query_field query_vectorlimitr0zSearch failed: r1rIr2rccs tdfddDdS)NrAcsi|]}|vr||qSr#r#)rDrW) exclude_keysitemr#r$ sz?Epsilla.similarity_search....) page_contentrVr rrrqrur$s  z+Epsilla.similarity_search..r3) r rr]rrmr5r6r7r8r;map)r!rmrnr)r\ror9r:r#rvr$similarity_searchs&   zEpsilla.similarity_searchz Type[Epsilla]) clsrZ embeddingr<rrrr)r[r\r&c Ks.t||||d} | j|f|||d| | S)aCreate an Epsilla vectorstore from raw documents. Args: texts (List[str]): List of text data to be inserted. embeddings (Embeddings): Embedding function. client (pyepsilla.vectordb.Client): Epsilla client to connect to. metadatas (Optional[List[dict]]): Metadata for each text. Defaults to None. db_path (Optional[str]): The path where the database will be persisted. Defaults to "/tmp/langchain-epsilla". db_name (Optional[str]): Give a name to the loaded database. Defaults to "langchain_store". collection_name (Optional[str]): Which collection to use. Defaults to "langchain_collection". If provided, default collection name will be set as well. drop_old (Optional[bool]): Whether to drop the previous collection and create a new one. Defaults to False. Returns: Epsilla: Epsilla vector store. )rr)r<r)r[)rrk) rzrZr{r<rrrr)r[r\instancer#r#r$ from_textss"zEpsilla.from_texts) rz documentsr{rrrr)r[r\r&c Ks>dd|D} dd|D} |j| |f| |||||d|S)a"Create an Epsilla vectorstore from a list of documents. Args: texts (List[str]): List of text data to be inserted. embeddings (Embeddings): Embedding function. client (pyepsilla.vectordb.Client): Epsilla client to connect to. metadatas (Optional[List[dict]]): Metadata for each text. Defaults to None. db_path (Optional[str]): The path where the database will be persisted. Defaults to "/tmp/langchain-epsilla". db_name (Optional[str]): Give a name to the loaded database. Defaults to "langchain_store". collection_name (Optional[str]): Which collection to use. Defaults to "langchain_collection". If provided, default collection name will be set as well. drop_old (Optional[bool]): Whether to drop the previous collection and create a new one. Defaults to False. Returns: Epsilla: Epsilla vector store. cSsg|] }|jqSr#)rtrDdocr#r#r$rFmrGz*Epsilla.from_documents..cSsg|] }|jqSr#)rVrr#r#r$rFnrG)r<rrrr)r[)r}) rzr~r{rrrr)r[r\rZr<r#r#r$from_documentsLs! zEpsilla.from_documents)r,)r,N)N)Nr,F)rlr,)__name__ __module__ __qualname____doc__r__annotations__rrr%propertyrr+r-r4rYrkry classmethodr}rr#r#r#r$rsH      /G(&,r)r __future__rloggingr`typingrrrrrrZlangchain_core.documentsr Zlangchain_core.embeddingsr Zlangchain_core.vectorstoresr rr getLoggerr5rr#r#r#r$s