a bgx?@sdZddlZddlmZmZmZmZddlZddlZddl m Z ddl m Z m Z mZmZmZedddZGd d d eZGd d d e ZdS) aaUtility for using SearxNG meta search API. SearxNG is a privacy-friendly free metasearch engine that aggregates results from `multiple search engines `_ and databases and supports the `OpenSearch `_ specification. More details on the installation instructions `here. <../../integrations/searx.html>`_ For the search API refer to https://docs.searxng.org/dev/search_api.html Quick Start ----------- In order to use this utility you need to provide the searx host. This can be done by passing the named parameter :attr:`searx_host ` or exporting the environment variable SEARX_HOST. Note: this is the only required parameter. Then create a searx search instance like this: .. code-block:: python from langchain_community.utilities import SearxSearchWrapper # when the host starts with `http` SSL is disabled and the connection # is assumed to be on a private network searx_host='http://self.hosted' search = SearxSearchWrapper(searx_host=searx_host) You can now use the ``search`` instance to query the searx API. Searching --------- Use the :meth:`run() ` and :meth:`results() ` methods to query the searx API. Other methods are available for convenience. :class:`SearxResults` is a convenience wrapper around the raw json result. Example usage of the ``run`` method to make a search: .. code-block:: python s.run(query="what is the best search engine?") Engine Parameters ----------------- You can pass any `accepted searx search API `_ parameters to the :py:class:`SearxSearchWrapper` instance. In the following example we are using the :attr:`engines ` and the ``language`` parameters: .. code-block:: python # assuming the searx host is set as above or exported as an env variable s = SearxSearchWrapper(engines=['google', 'bing'], language='es') Search Tips ----------- Searx offers a special `search syntax `_ that can also be used instead of passing engine parameters. For example the following query: .. code-block:: python s = SearxSearchWrapper("langchain library", engines=['github']) # can also be written as: s = SearxSearchWrapper("langchain library !github") # or even: s = SearxSearchWrapper("langchain library !gh") In some situations you might want to pass an extra string to the search query. For example when the `run()` method is called by an agent. The search suffix can also be used as a way to pass extra parameters to searx or the underlying search engines. .. code-block:: python # select the github engine and pass the search suffix s = SearchWrapper("langchain library", query_suffix="!gh") s = SearchWrapper("langchain library") # select github the conventional google search syntax s.run("large language models", query_suffix="site:github.com") *NOTE*: A search suffix can be defined on both the instance and the method level. The resulting query will be the concatenation of the two with the former taking precedence. See `SearxNG Configured Engines `_ and `SearxNG Search Syntax `_ for more details. Notes ----- This wrapper is based on the SearxNG fork https://github.com/searxng/searxng which is better maintained than the original Searx project and offers more features. Public searxNG instances often use a rate limiter for API usage, so you might want to use a self hosted instance and disable the rate limiter. If you are self-hosting an instance you can customize the rate limiter for your own network as described `here `_. For a list of public SearxNG instances see https://searx.space/ N)AnyDictListOptional)get_from_dict_or_env) BaseModel ConfigDictField PrivateAttrmodel_validatorreturncCs dddS)Nenjson)languageformatrrrx/var/www/html/cobodadashboardai.evdpl.com/venv/lib/python3.9/site-packages/langchain_community/utilities/searx_search.py_get_default_paramssrcsfeZdZUdZdZeed<edfdd Zeddd Ze e dd d Z e e dd d Z Z S) SearxResultsz,Dict like wrapper around search api results._data)datacs t|}t|||_dS)zATake a raw result from Searx and make it into a dict like object.N)rloadssuper__init____dict__)selfrZ json_data __class__rrrs  zSearxResults.__init__r cCs|jS)z$Text representation of searx result.)rrrrr__str__szSearxResults.__str__cCs |dS)zGSilence mypy for accessing this field. :meta private: resultsgetr rrrr"szSearxResults.resultscCs |dS)z#Helper accessor on the json result.answersr#r rrrr%szSearxResults.answers)__name__ __module__ __qualname____doc__rstr__annotations__rr!propertyrr"r% __classcell__rrrrrs  rc @seZdZUdZeZeed<dZe ed<dZ e ed<e e dZeed<d Zeeed <gZeee ed <gZeee ed <dZee ed <dZeed<d Zeeed<eddeeedddZeddZeedddZ eedddZ!d)e eee eee ee ee dddZ"d*e eee ee ee d d!d"Z#d+e eeee eee ee eeed#d$d%Z$d,e eeee ee eeed&d'd(Z%d S)-SearxSearchWrapperaWrapper for Searx API. To use you need to provide the searx host by passing the named parameter ``searx_host`` or exporting the environment variable ``SEARX_HOST``. In some situations you might want to disable SSL verification, for example if you are running searx locally. You can do this by passing the named parameter ``unsecure``. You can also pass the host url scheme as ``http`` to disable SSL. Example: .. code-block:: python from langchain_community.utilities import SearxSearchWrapper searx = SearxSearchWrapper(searx_host="http://localhost:8888") Example with SSL disabled: .. code-block:: python from langchain_community.utilities import SearxSearchWrapper # note the unsecure parameter is not needed if you pass the url scheme as # http searx = SearxSearchWrapper(searx_host="http://localhost:8888", unsecure=True) _resultr searx_hostFunsecure)default_factoryparamsNheadersengines categories query_suffix k aiosessionbefore)mode)valuesr cCs|di}t}i|||d<|d}|rBd||dd<|d}|rbd||dd<t|dd}|dstd|d d |}n|d rd |d <||d<|S)z?Validate that custom searx params are merged with default ones.r3r5,r6r0Z SEARX_HOSThttpzRWarning: missing the url scheme on host ! assuming secure https:// zhttps://zhttp://Tr1)r$rjoinr startswithprint)clsr=Z user_paramsdefaultr5r6r0rrrvalidate_paramss*       z"SearxSearchWrapper.validate_paramsZforbid)extra)r3r cCs@tj|j|j||j d}|js,td|jt|j}||_ |S)zActual request to searx API.r4r3verifySearx API returned an error: ) requestsr$r0r4r1ok ValueErrortextrr/)rr3Z raw_resultresrrr_searx_api_querys  z#SearxSearchWrapper._searx_api_queryc sT|jst4IdH}|j|d}|jr2d|d<|j|jfi|4IdH@}|jsbtd|j t | IdH}||_ WdIdHq1IdHs0YWdIdHq1IdHs0Yn|jj|j|j||j d4IdHD}|js td|j t | IdH}||_ WdIdHqP1IdHsF0Y|S)N)r4r3FsslrJrH) r:aiohttpZ ClientSessionr4r1r$r0rLrMrNrr/)rr3sessionkwargsresponseresultrrr_asearx_api_query s0 \ 4z$SearxSearchWrapper._asearx_api_query)queryr5r6r7rTr c Ksd|i}i|j||}|jrDt|jdkrD|dd|j7<t|trnt|dkrn|dd|7<t|trt|dkrd||d<t|trt|dkrd||d<||}t|jdkr|jd} n6t|j dkr ddd |j d |j D} nd } | S) aoRun query through Searx API and parse results. You can pass any other params to the searx query API. Args: query: The query to search for. query_suffix: Extra suffix appended to the query. engines: List of engines to use for the query. categories: List of categories to use for the query. **kwargs: extra parameters to pass to the searx API. Returns: str: The result of the query. Raises: ValueError: If an error occurred with the query. Example: This will make a query to the qwant engine: .. code-block:: python from langchain_community.utilities import SearxSearchWrapper searx = SearxSearchWrapper(searx_host="http://my.searx.host") searx.run("what is the weather in France ?", engine="qwant") # the same result can be achieved using the `!` syntax of searx # to select the engine using `query_suffix` searx.run("what is the weather in France ?", query_suffix="!qwant") qrr@r>r5r6 cSsg|]}|ddqScontentrr#.0rrrr dz*SearxSearchWrapper.run..NNo good search result found) r3r7len isinstancer*listrArPr%r"r9) rrXr5r6r7rT_paramsr3rOtoretrrrrun%s$(  "zSearxSearchWrapper.run)rXr5r7rTr c sd|i}i|j||}|jrDt|jdkrD|dd|j7<t|trnt|dkrn|dd|7<t|trt|dkrd||d<||IdH}t|jdkr|jd}n4t|j dkrddd |j d|j D}nd }|S) z Asynchronously version of `run`.rYrr@r>r5NrZcSsg|]}|ddqSr[r#r]rrrr`raz+SearxSearchWrapper.arun..rb) r3r7rcrdr*rerArWr%r"r9) rrXr5r7rTrfr3rOrgrrrarunjs  "zSearxSearchWrapper.arun)rX num_resultsr5r6r7rTr c Ksd|i}i|j||}|jrDt|jdkrD|dd|j7<t|trnt|dkrn|dd|7<t|trt|dkrd||d<t|trt|dkrd||d<||jd|} t| dkrdd igSd d | DS) a$Run query through Searx API and returns the results with metadata. Args: query: The query to search for. query_suffix: Extra suffix appended to the query. num_results: Limit the number of results to return. engines: List of engines to use for the query. categories: List of categories to use for the query. **kwargs: extra parameters to pass to the searx API. Returns: Dict with the following keys: { snippet: The description of the result. title: The title of the result. link: The link to the result. engines: The engines used for the result. category: Searx category of the result. } rYrr@r>r5r6NResultNo good Search Result was foundcSs4g|],}|dd|d|d|d|ddqSr\rtitleurlr5category)Zsnippetrnlinkr5rpr#r^rVrrrr`s z.SearxSearchWrapper.results..) r3r7rcrdr*rerArPr") rrXrjr5r6r7rTrfr3r"rrrr"s"  zSearxSearchWrapper.results)rXrjr5r7rTr c sd|i}i|j||}|jrDt|jdkrD|dd|j7<t|trnt|dkrn|dd|7<t|trt|dkrd||d<||IdHjd|}t|dkrddigSd d |DS) zdAsynchronously query with json results. Uses aiohttp. See `results` for more info. rYrr@r>r5NrkrlcSs4g|],}|dd|d|d|d|ddqSrmr#rrrrrr`s z/SearxSearchWrapper.aresults..) r3r7rcrdr*rerArWr") rrXrjr5r7rTrfr3r"rrraresultss   zSearxSearchWrapper.aresults)NNr)Nr)NNr)Nr)&r&r'r(r)r r/rr+r0r*r1boolr rr3dictr4rr5rr6r7r9intr:rr classmethodrrFrZ model_configrPrWrhrir"rsrrrrr.sz       H  '   = r.)r)rtypingrrrrrRrKZlangchain_core.utilsrZpydanticrrr r r rurrr.rrrrs