a bg@sldZddlZddlmZmZmZmZddlZddlZddl m Z ddl m Z m Z mZdZGddde ZdS) zWUtil that calls Metaphor Search API. In order to set this up, follow instructions at: N)AnyDictListOptionalget_from_dict_or_env) BaseModel ConfigDictmodel_validatorzhttps://api.metaphor.systemsc @sDeZdZUdZeed<dZeed<eddZ deee e ee e ee ee ee ee ee e e e d d d Zed d eeed ddZdeee e ee e ee ee ee ee ee e e ed ddZdeee e ee e ee ee ee ee ee e e ed ddZe ee edddZdS)MetaphorSearchAPIWrapperz Wrapper for Metaphor Search API.metaphor_api_key kZforbid)extraN) query num_resultsinclude_domainsexclude_domainsstart_crawl_dateend_crawl_datestart_published_dateend_published_dateuse_autopromptreturnc  CsPd|ji} ||||||||| d } tjtd| | d} | | } | dS)N X-Api-Key Z numResultsrZincludeDomainsZexcludeDomainsZstartCrawlDateZ endCrawlDateZstartPublishedDateZendPublishedDateZ useAutoprompt/search)headersjsonresults)r requestspostMETAPHOR_API_URLraise_for_statusr)selfrrrrrrrrrrparamsresponseZsearch_resultsr'{/var/www/html/cobodadashboardai.evdpl.com/venv/lib/python3.9/site-packages/langchain_community/utilities/metaphor_search.py_metaphor_search_resultss&  z1MetaphorSearchAPIWrapper._metaphor_search_resultsbefore)mode)valuesrcCst|dd}||d<|S)z9Validate that api key and endpoint exists in environment.r ZMETAPHOR_API_KEYr)clsr,r r'r'r(validate_environment>s z-MetaphorSearchAPIWrapper.validate_environmentc Cs&|j||||||||| d } || S)aLRun query through Metaphor Search and return metadata. Args: query: The query to search for. num_results: The number of results to return. include_domains: A list of domains to include in the search. Only one of include_domains and exclude_domains should be defined. exclude_domains: A list of domains to exclude from the search. Only one of include_domains and exclude_domains should be defined. start_crawl_date: If specified, only pages we crawled after start_crawl_date will be returned. end_crawl_date: If specified, only pages we crawled before end_crawl_date will be returned. start_published_date: If specified, only pages published after start_published_date will be returned. end_published_date: If specified, only pages published before end_published_date will be returned. use_autoprompt: If true, we turn your query into a more Metaphor-friendly query. Adds latency. Returns: A list of dictionaries with the following keys: title - The title of the page url - The url author - Author of the content, if applicable. Otherwise, None. published_date - Estimated date published in YYYY-MM-DD format. Otherwise, None. )rrrrrrrr)r)_clean_results) r$rrrrrrrrrraw_search_resultsr'r'r(rIs! z MetaphorSearchAPIWrapper.resultsc sHtd f dd } | IdH} t| } | dS)z8Get results from the Metaphor Search API asynchronously.)rc s dji} d }t4IdH}|jtd||d4IdHr}|jdkr|IdH}|WdIdHWdIdHStd|jd|jWdIdHq1IdHs0YWdIdHq1IdHs0YdS)Nrrr)rrzError z: ) r aiohttpZ ClientSessionr!r"statustext Exceptionreason)rr%sessionresdata rrrrrrr$rrrr'r(fetchs(    ,z5MetaphorSearchAPIWrapper.results_async..fetchNr)strrloadsr/) r$rrrrrrrrrr;Zresults_json_strZ results_jsonr'r:r( results_asyncws$  z&MetaphorSearchAPIWrapper.results_async)r0rc CsFg}|D]8}||dd|dd|dd|ddd q|S) Ntitlez Unknown Titleurlz Unknown URLauthorzUnknown AuthorZ publishedDatez Unknown Date)r?r@rAZpublished_date)appendget)r$r0Zcleaned_resultsresultr'r'r(r/s    z'MetaphorSearchAPIWrapper._clean_results)NNNNNNN)NNNNNNN)NNNNNNN)__name__ __module__ __qualname____doc__r<__annotations__rintr Z model_configrrbooldictr)r classmethodrrr.rr>r/r'r'r'r(r s     #    2   *r )rHrtypingrrrrr2r Zlangchain_core.utilsrZpydanticrr r r"r r'r'r'r(s