a bg@sddlZddlmZddlmZddlmZmZmZm Z m Z m Z m Z ddl mZddlmZddlmZddlmZmZGd d d eZGd d d eZdS) N) TextIOWrapper)Path)AnyDictIteratorListOptionalSequenceUnion)Document) BaseLoader)detect_file_encodings)UnstructuredFileLoadervalidate_unstructured_versionc @steZdZdZdddeeefeeeeee eee eedddZ e e d d d Zee e d d dZdS) CSVLoaderaLoad a `CSV` file into a list of Documents. Each document represents one row of the CSV file. Every row is converted into a key/value pair and outputted to a new line in the document's page_content. The source for each document loaded from csv is set to the value of the `file_path` argument for all documents by default. You can override this by setting the `source_column` argument to the name of a column in the CSV file. The source of each document will then be set to the value of the column with the name specified in `source_column`. Output Example: .. code-block:: txt column1: value1 column2: value2 column3: value3 Instantiate: .. code-block:: python from langchain_community.document_loaders import CSVLoader loader = CSVLoader(file_path='./hw_200.csv', csv_args={ 'delimiter': ',', 'quotechar': '"', 'fieldnames': ['Index', 'Height', 'Weight'] }) Load: .. code-block:: python docs = loader.load() print(docs[0].page_content[:100]) print(docs[0].metadata) .. code-block:: python Index: Index Height: Height(Inches)" Weight: "Weight(Pounds)" {'source': './hw_200.csv', 'row': 0} Async load: .. code-block:: python docs = await loader.aload() print(docs[0].page_content[:100]) print(docs[0].metadata) .. code-block:: python Index: Index Height: Height(Inches)" Weight: "Weight(Pounds)" {'source': './hw_200.csv', 'row': 0} Lazy load: .. code-block:: python docs = [] docs_lazy = loader.lazy_load() # async variant: # docs_lazy = await loader.alazy_load() for doc in docs_lazy: docs.append(doc) print(docs[0].page_content[:100]) print(docs[0].metadata) .. code-block:: python Index: Index Height: Height(Inches)" Weight: "Weight(Pounds)" {'source': './hw_200.csv', 'row': 0} NF)content_columns) file_path source_columnmetadata_columnscsv_argsencodingautodetect_encodingrcCs2||_||_||_||_|pi|_||_||_dS)a Args: file_path: The path to the CSV file. source_column: The name of the column in the CSV file to use as the source. Optional. Defaults to None. metadata_columns: A sequence of column names to use as metadata. Optional. csv_args: A dictionary of arguments to pass to the csv.DictReader. Optional. Defaults to None. encoding: The encoding of the CSV file. Optional. Defaults to None. autodetect_encoding: Whether to try to autodetect the file encoding. content_columns: A sequence of column names to use for the document content. If not present, use all columns that are not part of the metadata. N)rrrrrrr)selfrrrrrrrrr}/var/www/html/cobodadashboardai.evdpl.com/venv/lib/python3.9/site-packages/langchain_community/document_loaders/csv_loader.py__init__cs zCSVLoader.__init__returnc csFzFt|jd|jd }||EdHWdn1s:0YWnty }z|jrt|j}|D]t}zXt|jd|jd2}||EdHWdWqWdn1s0YWqltyYqlYql0qlntd|j|WYd}~n@d}~0ty@}ztd|j|WYd}~n d}~00dS)N)newlinerzError loading ) openrr_CSVLoader__read_fileUnicodeDecodeErrorrr RuntimeError Exception)rcsvfileeZdetected_encodingsrrrr lazy_loads&2  4 &zCSVLoader.lazy_load)r%rc #stj|fij}t|D]\}}z"jdur:|jntj}Wn$tyjtdjdYn0d fdd| D}||d}j D]8}z||||<Wqtytd|dYq0qt ||dVqdS) NzSource column 'z' not found in CSV file. c3sz|]r\}}jr|jvrn |jvr|dur6|n|dt|trN|nt|trjdttj|n|VqdS)Nz: ,)rrstrip isinstancestrlistjoinmap).0kvrrr s   z(CSVLoader.__read_file..)sourcerowzMetadata column ')Z page_contentmetadata) csv DictReaderr enumeraterr,rKeyError ValueErrorr.itemsrr ) rr%Z csv_readerir6r5contentr7colrr3rZ __read_files*      zCSVLoader.__read_file)NrNNF)__name__ __module__ __qualname____doc__r r,rrr rboolrrr r'rr!rrrrrs&U   !rcs:eZdZdZd eeedfdd ZedddZZ S) UnstructuredCSVLoadera|Load `CSV` files using `Unstructured`. Like other Unstructured loaders, UnstructuredCSVLoader can be used in both "single" and "elements" mode. If you use the loader in "elements" mode, the CSV file will be a single Unstructured Table element. If you use the loader in "elements" mode, an HTML representation of the table will be available in the "text_as_html" key in the document metadata. Examples -------- from langchain_community.document_loaders.csv_loader import UnstructuredCSVLoader loader = UnstructuredCSVLoader("stanley-cups.csv", mode="elements") docs = loader.load() single)rmodeunstructured_kwargsc s&tddtjf||d|dS)a Args: file_path: The path to the CSV file. mode: The mode to use when loading the CSV file. Optional. Defaults to "single". **unstructured_kwargs: Keyword arguments to pass to unstructured. z0.6.8)Zmin_unstructured_version)rrHN)rsuperr)rrrHrI __class__rrrs zUnstructuredCSVLoader.__init__rcCs"ddlm}|fd|ji|jS)Nr) partition_csvfilename)Zunstructured.partition.csvrMrrI)rrMrrr _get_elementss z#UnstructuredCSVLoader._get_elements)rG) rArBrCrDr,rrrrO __classcell__rrrKrrFs rF)r8iorpathlibrtypingrrrrrr r Zlangchain_core.documentsr Z)langchain_community.document_loaders.baser Z,langchain_community.document_loaders.helpersr Z1langchain_community.document_loaders.unstructuredrrrrFrrrrs  $   +