a bg @sVddlmZmZmZmZddlmZddlmZm Z ddl m Z Gddde eZ dS))AnyCallableDictList)Document) BaseModelmodel_validator) BaseLoadercseZdZUdZeed<eed<eege fed<eeege fdfdd Z e dd e eed d d Z ee d ddZZS)ApifyDatasetLoaderaLoad datasets from `Apify` web scraping, crawling, and data extraction platform. For details, see https://docs.apify.com/platform/integrations/langchain Example: .. code-block:: python from langchain_community.document_loaders import ApifyDatasetLoader from langchain_core.documents import Document loader = ApifyDatasetLoader( dataset_id="YOUR-DATASET-ID", dataset_mapping_function=lambda dataset_item: Document( page_content=dataset_item["text"], metadata={"source": dataset_item["url"]} ), ) documents = loader.load() apify_client dataset_iddataset_mapping_functionr r cstj||ddS)arInitialize the loader with an Apify dataset ID and a mapping function. Args: dataset_id (str): The ID of the dataset on the Apify platform. dataset_mapping_function (Callable): A function that takes a single dictionary (an Apify dataset item) and converts it to an instance of the Document class. rN)super__init__)selfr r  __class__/var/www/html/cobodadashboardai.evdpl.com/venv/lib/python3.9/site-packages/langchain_community/document_loaders/apify_dataset.pyr%s zApifyDatasetLoader.__init__before)mode)valuesreturncCs`z@ddlm}|}t|jd}r6|jdd7<||d<WntyZtdYn0|S)zYValidate environment. Args: values: The values to validate. r) ApifyClient httpx_clientz user-agentz; Origin/langchainr z`Could not import apify-client Python package. Please install it with `pip install apify-client`.)r rgetattr http_clientheaders ImportError)clsrrclientrrrrvalidate_environment4s     z'ApifyDatasetLoader.validate_environment)rcCs(|j|jjddj}tt|j|S)zLoad documents.T)clean)r Zdatasetr Z list_itemsitemslistmapr )rZ dataset_itemsrrrloadMszApifyDatasetLoader.load)__name__ __module__ __qualname____doc__r__annotations__strrrrrr classmethodr"rr' __classcell__rrrrr s r N) typingrrrrZlangchain_core.documentsrZpydanticrrZ)langchain_community.document_loaders.baser r rrrrs