a bg@sDddlmZmZmZmZddlmZddlmZGdddeZ dS))DictIteratorListOptional) BaseLoader)DocumentcseZdZdZdeeeeddfdd ZddddZeeefdd d d Z e e dd d Z e e dddZ ee dddZZS) NeedleLoadera NeedleLoader is a document loader for managing documents stored in a collection. Setup: Install the `needle-python` library and set your Needle API key. .. code-block:: bash pip install needle-python export NEEDLE_API_KEY="your-api-key" Key init args: - `needle_api_key` (Optional[str]): API key for authenticating with Needle. - `collection_id` (str): Needle collection to load documents from. Usage: .. code-block:: python from langchain_community.document_loaders.needle import NeedleLoader loader = NeedleLoader( needle_api_key="your-api-key", collection_id="your-collection-id" ) # Load documents documents = loader.load() for doc in documents: print(doc.metadata) # Lazy load documents for doc in loader.lazy_load(): print(doc.metadata) N)needle_api_key collection_idreturncsnzddlm}Wnty*tdYn0t||_||_d|_|jr\||jd|_|jsjtddS)a Initializes the NeedleLoader with API key and collection ID. Args: needle_api_key (Optional[str]): API key for authenticating with Needle. collection_id (Optional[str]): Identifier for the Needle collection. Raises: ImportError: If the `needle-python` library is not installed. ValueError: If the collection ID is not provided. r) NeedleClientzDPlease install with `pip install needle-python` to use NeedleLoader.N)Zapi_keyCollection ID must be provided.) Z needle.v1r ImportErrorsuper__init__r r client ValueError)selfr r r  __class__y/var/www/html/cobodadashboardai.evdpl.com/venv/lib/python3.9/site-packages/langchain_community/document_loaders/needle.pyr+s   zNeedleLoader.__init__)r cCs$|jdurtd|js tddS)z Ensures the Needle collection is set and the client is initialized. Raises: ValueError: If the Needle client is not initialized or if the collection ID is missing. Nz9NeedleClient is not initialized. Provide a valid API key.r )rrr rrrr_get_collectionMs  zNeedleLoader._get_collection)filesr csvzddlmWnty*tdYn0||jdusFJdfdd|D}|jjjj|j |ddS) an Adds files to the Needle collection. Args: files (Dict[str, str]): Dictionary where keys are file names and values are file URLs. Raises: ImportError: If the `needle-python` library is not installed. ValueError: If the collection is not properly initialized. r FileToAddz=Please install with `pip install needle-python` to add files.N!NeedleClient must be initialized.csg|]\}}||dqS))nameurlr).0rrrrr rz*NeedleLoader.add_files..)r r) Zneedle.v1.modelsrrrritems collectionsraddr )rrZ files_to_addrrr add_files\s    zNeedleLoader.add_filescCs>||jdusJd|jjj|j}dd|D}|S)a  Fetches metadata for documents from the Needle collection. Returns: List[Document]: A list of documents with metadata. Content is excluded. Raises: ValueError: If the collection is not properly initialized. Nrc Ss6g|].}|jdkrtd|j|jt|ddddqS)ZindexedsizeN)sourcetitler()Z page_contentmetadata)statusrrrgetattr)r filerrrr!s   z1NeedleLoader._fetch_documents..)rrr$rlistr )rrdocsrrr_fetch_documentsxs   zNeedleLoader._fetch_documentscCs|S)z Loads all documents from the Needle collection. Returns: List[Document]: A list of documents from the collection. r1rrrrloadszNeedleLoader.loadccs|EdHdS)z Lazily loads documents from the Needle collection. Yields: Iterator[Document]: An iterator over the documents. Nr2rrrr lazy_loadszNeedleLoader.lazy_load)NN)__name__ __module__ __qualname____doc__rstrrrrr&rrr1r3rr4 __classcell__rrrrrs%" rN) typingrrrrZ$langchain_core.document_loaders.baserZlangchain_core.documentsrrrrrrs