a bg@sfddlZddlmZmZmZmZmZddlmZddl m Z ddl m Z e eZGddde ZdS)N)AnyIteratorListOptionalSequence)Document) BaseLoader) NewsURLLoaderc@speZdZdZdeeeeeeeeddddZ e e dd d Z e eedd d Zee dd dZdS) RSSFeedLoaderaQLoad news articles from `RSS` feeds using `Unstructured`. Args: urls: URLs for RSS feeds to load. Each articles in the feed is loaded into its own document. opml: OPML file to load feed urls from. Only one of urls or opml should be provided. The value can be a URL string, or OPML markup contents as byte or string. continue_on_failure: If True, continue loading documents even if loading fails for a particular URL. show_progress_bar: If True, use tqdm to show a loading progress bar. Requires tqdm to be installed, ``pip install tqdm``. **newsloader_kwargs: Any additional named arguments to pass to NewsURLLoader. Example: .. code-block:: python from langchain_community.document_loaders import RSSFeedLoader loader = RSSFeedLoader( urls=["", ""], ) docs = loader.load() The loader uses feedparser to parse RSS feeds. The feedparser library is not installed by default so you should install it if using this loader: https://pythonhosted.org/feedparser/ If you use OPML, you should also install listparser: https://pythonhosted.org/listparser/ Finally, newspaper is used to process each article: https://newspaper.readthedocs.io/en/latest/ NTF)urlsopmlcontinue_on_failureshow_progress_barnewsloader_kwargsreturncKs:|du|dukrtd||_||_||_||_||_dS)zInitialize with urls or OPML.Nz;Provide either the urls or the opml argument, but not both.) ValueErrorr r r rr)selfr r r rrrv/var/www/html/cobodadashboardai.evdpl.com/venv/lib/python3.9/site-packages/langchain_community/document_loaders/rss.py__init__/s zRSSFeedLoader.__init__)rc Cs^|}|jrVzddlm}Wn.tyL}ztd|WYd}~n d}~00||}t|S)Nr)tqdmzPackage tqdm must be installed if show_progress_bar=True. Please install with 'pip install tqdm' or set show_progress_bar=False.) lazy_loadrr ImportErrorlist)riterrerrrloadDszRSSFeedLoader.loadc Csd|jr |jSz ddl}Wn.tyF}ztd|WYd}~n d}~00||j}dd|jDS)NrzPackage listparser must be installed if the opml arg is used. Please install with 'pip install listparser' or use the urls arg instead.cSsg|] }|jqSr)url).0feedrrr _z+RSSFeedLoader._get_urls..)r listparserrparser Zfeeds)rr"rZrssrrr _get_urlsRs  zRSSFeedLoader._get_urlsc cs\z ddl}Wnty&tdYn0|jD]&}z0||}t|ddrbtd|d|jWnRty}z:|jrt d|d|WYd}~q.n|WYd}~n d}~00zD|j D]8}t fd|j gi|j}|d}||jd<|VqWq.tyT}z>|jrs