-
Notifications
You must be signed in to change notification settings - Fork 2.3k
Description
Its well known that qwant blocks very fast requests from a SearXNG instance:
We already discussed this issue more than a year ago in #2011 (comment) and the documentation was a result of the issue:
This method no longer works; the captcha dialog is not displayed in the WEB browser session. This leads me to believe that qwant has changed its procedure for bot recognition and we should re-engineer the engine again.
I don't know if it might be related to bot-detection; I noticed that qwant makes requests to the server in the browser that look like a ping to me. The returned response has a size of 0 Bytes
In this request a JSON structure is send to the qwant server:
This search_id comes from the previously sent query / e.g.
In the response to our query, beside the result items, we find this search_id
To be clear, first the query is send (GET https://api.qwant.com/v3/search/web?), in the response there is a search_id and this search ID is pinged back to https://www.qwant.com/action/display (POST)
Since https://www.qwant.com/action/display is not a part of the API (api.qwant.com/v3) and the response is always 0 Byte I have the assumption that this could be related to the bot detection of qwant ... would have to be investigated further to validate this assumption.
Related: