Video (only) thumbnails not always rendered #3924

jcarnat · 2024-10-15T10:41:25Z

jcarnat
Oct 15, 2024

Version of SearXNG, commit number if you are using on master branch and stipulate if you forked SearXNG

I can trigger this with 2024.10.10+81aaca8f4 and 2024.10.15+3e87354f0.

How did you install SearXNG?

It is manually installed accordingly to the wiki.
It runs on OpenBSD 7.6. Using Python 3.11 venv and uWSGI 2.0.27.
It is exposed via OpenBSD relayd. But it happens both with and without it; that is when I point my web browser at uwsgi http port.

What happened?

When searching in Videos section, some results don't display thumbnails.

Most of the time, those are Peertube instances results. But some instance gets thumbnails, some don't.
I think YouTube video always get a thumbnail.

Using Shift-Reload on the web browser doesn't load missing thumbnails.

I have checked my DNS and system logs but there was nothing that indicates a failure in DNS or a raised system limit.

How To Reproduce

Search whatever in the Videos section.

I could reproduce it on public searxng instance when enabling Peertube/Sepiasearch egines. Although it happens in less cases.

Expected behavior

Get thumbnails for every videos.

Screenshots & Logs

Here's what the logs says:

2024-10-15 12:21:06,184 ERROR:searx.webapp: HTTP error                                                              
Traceback (most recent call last):                                                                                                                                                                                                      
  File "/home/searxng-blue/pyenv/lib/python3.11/site-packages/anyio/_core/_sockets.py", line 187, in connect_tcp                                                                                                                        
    addr_obj = ip_address(remote_host)                                                                                                                                                                                                  
               ^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                                                  
  File "/usr/local/lib/python3.11/ipaddress.py", line 54, in ip_address
    raise ValueError(f'{address!r} does not appear to be an IPv4 or IPv6 address')                                  
ValueError: 'videos.pair2jeux.tube' does not appear to be an IPv4 or IPv6 address                                   
                                                                                                                    
During handling of the above exception, another exception occurred:                                
                                                                                                                    
Traceback (most recent call last):                                                                                  
  File "/home/searxng-blue/pyenv/lib/python3.11/site-packages/httpcore/_exceptions.py", line 10, in map_exceptions  
    yield                                                                                                                                                                                                                               
  File "/home/searxng-blue/pyenv/lib/python3.11/site-packages/httpcore/_backends/anyio.py", line 114, in connect_tcp 
    stream: anyio.abc.ByteStream = await anyio.connect_tcp(                                                                                                                                                                             
                                   ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/searxng-blue/pyenv/lib/python3.11/site-packages/anyio/_core/_sockets.py", line 190, in connect_tcp    
    gai_res = await getaddrinfo(                                                                                    
              ^^^^^^^^^^^^^^^^^^                                                                                                                                                                                                        
socket.gaierror: [Errno -5] no address associated with name                                                                                                                                                                             
                                                                                                                                                                                                                                        
The above exception was the direct cause of the following exception:                                                
                                                                                                                    
Traceback (most recent call last):                                                                                  
  File "/home/searxng-blue/pyenv/lib/python3.11/site-packages/httpx/_transports/default.py", line 60, in map_httpcore_exceptions
    yield                                                                                                                                                                                                                               
  File "/home/searxng-blue/pyenv/lib/python3.11/site-packages/httpx/_transports/default.py", line 353, in handle_async_request
    resp = await self._pool.handle_async_request(req)                                                                                                                                                                                   
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                               
  File "/home/searxng-blue/pyenv/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 262, in handle_async_request                                                                                                    
    raise exc                                         
  File "/home/searxng-blue/pyenv/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 245, in handle_async_request                                                                                                    
    response = await connection.handle_async_request(request)                                                                                                                                                                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                             File "/home/searxng-blue/pyenv/lib/python3.11/site-packages/httpcore/_async/connection.py", line 92, in handle_async_request                                                                                                          
    raise exc                                                                                                                                                                                                                           
  File "/home/searxng-blue/pyenv/lib/python3.11/site-packages/httpcore/_async/connection.py", line 69, in handle_async_request                                                                                                          
    stream = await self._connect(request)                                                                                                                                                                                                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                           
  File "/home/searxng-blue/pyenv/lib/python3.11/site-packages/httpcore/_async/connection.py", line 117, in _connect                                                                                                                     
    stream = await self._network_backend.connect_tcp(**kwargs)                                                                                                                                                                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                          
  File "/home/searxng-blue/pyenv/lib/python3.11/site-packages/httpcore/_backends/auto.py", line 31, in connect_tcp                                                                                                                          return await self._backend.connect_tcp(                                                                         
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                                             
  File "/home/searxng-blue/pyenv/lib/python3.11/site-packages/httpcore/_backends/anyio.py", line 112, in connect_tcp                                                                                                                    
    with map_exceptions(exc_map):                                                                                   
  File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__                                             
    self.gen.throw(typ, value, traceback)                                                                           
  File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__                                                                                                                                                      [192/85725]
    self.gen.throw(typ, value, traceback)                                                                           
  File "/home/searxng-blue/pyenv/lib/python3.11/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions  
    raise to_exc(exc) from exc                                                                                                                                                                                                          
httpcore.ConnectError: [Errno -5] no address associated with name                                                                                                                                                                       
                                                                                                                    
The above exception was the direct cause of the following exception:                                                
                                                                                                                                                                                                                                        
Traceback (most recent call last):                                                                                                                                                                                                      
  File "/home/searxng/src/searx/webapp.py", line 1067, in image_proxy                                                                                                                                                                   
    resp, stream = http_stream(method='GET', url=url, headers=request_headers, allow_redirects=True)                                                                                                                                    
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                    
  File "/home/searxng/src/searx/network/__init__.py", line 258, in stream                                                                                                                                                               
    response = next(generator)  # pylint: disable=stop-iteration-return                                                                                                                                                                 
               ^^^^^^^^^^^^^^^                                                                                                                                                                                                          
  File "/home/searxng/src/searx/network/__init__.py", line 228, in _stream_generator                                                                                                                                                    
    raise obj_or_exception                                                                                          
  File "/home/searxng/src/searx/network/__init__.py", line 195, in stream_chunk_to_queue                                                                                                                                                
    async with await network.stream(method, url, **kwargs) as response:                                                                                                                                                                 
  File "/usr/local/lib/python3.11/contextlib.py", line 210, in __aenter__                                                                                                                                                               
    return await anext(self.gen)                                                                                                                                                                                                        
           ^^^^^^^^^^^^^^^^^^^^^                                                                                    
  File "/home/searxng-blue/pyenv/lib/python3.11/site-packages/httpx/_client.py", line 1573, in stream               
    response = await self.send(                                                                                     
               ^^^^^^^^^^^^^^^^                                                                                     
  File "/home/searxng-blue/pyenv/lib/python3.11/site-packages/httpx/_client.py", line 1617, in send
    response = await self._send_handling_auth(                                                                      
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                      
  File "/home/searxng-blue/pyenv/lib/python3.11/site-packages/httpx/_client.py", line 1645, in _send_handling_auth  
    response = await self._send_handling_redirects(                                                                                                                                                                                     
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                                     
  File "/home/searxng-blue/pyenv/lib/python3.11/site-packages/httpx/_client.py", line 1682, in _send_handling_redirects                                                                                                                 
    response = await self._send_single_request(request)                                                             
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                             
  File "/home/searxng-blue/pyenv/lib/python3.11/site-packages/httpx/_client.py", line 1719, in _send_single_request 
    response = await transport.handle_async_request(request)                                                                                                                                                                            
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                            
  File "/home/searxng-blue/pyenv/lib/python3.11/site-packages/httpx/_transports/default.py", line 352, in handle_async_request                                                                                                          
    with map_httpcore_exceptions():                                                                                 
  File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__                                             
    self.gen.throw(typ, value, traceback)                                                                           
  File "/home/searxng-blue/pyenv/lib/python3.11/site-packages/httpx/_transports/default.py", line 77, in map_httpcore_exceptions
    raise mapped_exc(message) from exc                                                                                                                                                                                                  
httpx.ConnectError: [Errno -5] no address associated with name                                                                                                                                                                          
[5561] 127.0.0.1 - - [15/Oct/2024:12:21:06 +0200] "GET /image_proxy?url=https%3A%2F%2Fvideos.pair2jeux.tube%2Flazy-static%2Fthumbnails%2F67fd82f5-3cb0-4253-90cf-7cc81f28dfd2.jpg&h=c6fefd4f19d5c120febe3dc69423c7b1b896cee69d9b9091157b
154a51bee8d4 HTTP/1.1" 400 1577 "-" "Mozilla/5.0 (Windows NT 10.0; rv:128.0) Gecko/20100101 Firefox/128.0"          
2024-10-15 12:21:06,191 WARNING:searx.limiter: PASS 127.0.0.1/32: matched PASSLIST - IP matches 127.0.0.1/32 in botdetection.ip_lists.pass_ip.

Additional context

Answered by jcarnat

Oct 15, 2024

I have redeployed a new test instance from scratch and the bug has disappeared.
The config has been copied from my problematic instance so I guess the problem lies somewhere in the Python venv...
Sorry for the noise!

View full answer

return42 · 2024-10-15T10:54:01Z

return42
Oct 15, 2024
Maintainer

httpx.ConnectError: [Errno -5] no address associated with name                                                                                                                                                                          
[5561] 127.0.0.1 - - [15/Oct/2024:12:21:06 +0200] "GET /image_proxy?url=https%3A%2F%2Fvideos.pair2jeux.tube%2Flazy-static%2Fthumbnails%2F67fd82f5-3cb0-4253-90cf-7cc81f28dfd2.jpg&h=c6fefd4f19d5c120febe3dc69423c7b1b896cee69d9b9091157b
154a51bee8d4 HTTP/1.1" 400 1577 "-" "Mozilla/5.0 (Windows NT 10.0; rv:128.0) Gecko/20100101 Firefox/128.0"

The image is loaded by the /image_proxy and the image-proxy tries to load the image from videos.pair2jeux .. but the DNS on your SearXNG does not return an IP for videos.pair2jeux.

From my IPs I can open: https://videos.pair2jeux.tube/lazy-static/thumbnails/67fd82f5-3cb0-4253-90cf-7cc81f28dfd2.jpg

0 replies

jcarnat · 2024-10-15T11:14:06Z

jcarnat
Oct 15, 2024
Author

The image is loaded by the /image_proxy and the image-proxy tries to load the image from videos.pair2jeux .. but the DNS on your SearXNG does not return an IP for videos.pair2jeux.

From my IPs I can open: https://videos.pair2jeux.tube/lazy-static/thumbnails/67fd82f5-3cb0-4253-90cf-7cc81f28dfd2.jpg

Hum… from the shell, it does resolve and I can wget the thumbnail. I’ll see if I can find something about DNS resolution in python venv and uwsgi documentation.

0 replies

return42 · 2024-10-15T12:46:01Z

return42
Oct 15, 2024
Maintainer

@jcarnat I move your issue to the discussions ..(its a bug of the infrastructure we can't solve in SearXNG core).

1 reply

jcarnat Oct 15, 2024
Author

Ok. I'll post about the solution if I can manage to find it :)

jcarnat · 2024-10-15T16:13:33Z

jcarnat
Oct 15, 2024
Author

I have redeployed a new test instance from scratch and the bug has disappeared.
The config has been copied from my problematic instance so I guess the problem lies somewhere in the Python venv...
Sorry for the noise!

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Video (only) thumbnails not always rendered #3924

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 4 comments 1 reply

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Video (only) thumbnails not always rendered #3924

Uh oh!

Uh oh!

jcarnat Oct 15, 2024

Replies: 4 comments · 1 reply

Uh oh!

Uh oh!

return42 Oct 15, 2024 Maintainer

Uh oh!

jcarnat Oct 15, 2024 Author

Uh oh!

return42 Oct 15, 2024 Maintainer

Uh oh!

jcarnat Oct 15, 2024 Author

Uh oh!

jcarnat Oct 15, 2024 Author

jcarnat
Oct 15, 2024

Replies: 4 comments 1 reply

return42
Oct 15, 2024
Maintainer

jcarnat
Oct 15, 2024
Author

return42
Oct 15, 2024
Maintainer

jcarnat Oct 15, 2024
Author

jcarnat
Oct 15, 2024
Author