Python app missing when deploying in TrueNAS Scale #3998

Karmalakas · 2024-11-06T13:37:44Z

Karmalakas
Nov 6, 2024

I reported this on the charts repo, but was redirected. Not sure if this is the right one, but here it goes:

Just tried adding SearXNG app and deployment fails with:

Startup probe failed: HTTP probe failed with statuscode: 500

Log is flooded with:
2024-11-05 11:04:13.507557+02:00--- no python application found, check your startup logs for errors ---
2024-11-05 11:04:13.507601+02:00[pid: 25|app: -1|req: -1/20] 172.16.0.1 () {26 vars in 313 bytes} [Tue Nov  5 09:04:13 2024] GET /healthz => generated 21 bytes in 0 msecs (HTTP/1.1 500) 3 headers in 102 bytes (0 switches on core 0)
TrueNAS Scale Dragonfish-24.04.2.3

Answered by Karmalakas

Nov 8, 2024

OK, so it seems it might've been my inexperience. Here's a copy of my comment on charts repo:

So I decided to try removing everything related to SearXNG and recreate from scratch. While creating a dataset, this time instead of choosing "Apps" preset, I left it on "Generic" - and it seems it worked. Everything loads and search itself runs, although in the logs I see this four times:

2024-11-08 16:38:48.319623+00:002024-11-08 16:38:48,319 WARNING:searx.botdetection.config: missing config file: /etc/searxng/limiter.toml

Anyway, considering this as resolved, but if you have any insight about this limiter.toml, I'd be grateful to hear it 🙏

View full answer

dalf · 2024-11-07T07:33:53Z

dalf
Nov 7, 2024

What are the first line of the log ?
Is the secret key setup in a way or another ?

I'm not familliar with charts, but I don't see how the SECRET_KEY is set looking at https://github.com/truenas/charts/tree/master/community/searxng

For reference, see:

3 replies

Karmalakas Nov 7, 2024
Author

I have some understanding, but I'm completely new to TrueNAS and charts and all... So please bear with me

Here's the full log:

2024-11-07 16:08:29.271530+02:00SearXNG version 
2024-11-07 16:08:29.273134+02:00Use existing /etc/searxng/uwsgi.ini
2024-11-07 16:08:29.273287+02:00Use existing /etc/searxng/settings.yml
2024-11-07 16:08:29.273300+02:00Listen on 0.0.0.0:30053
2024-11-07 16:08:29.303328+02:00[uWSGI] getting INI configuration from /etc/searxng/uwsgi.ini
2024-11-07 16:08:29.303358+02:00[uwsgi-static] added mapping for /static => /usr/local/searxng/searx/static
2024-11-07 16:08:29.305477+02:00*** Starting uWSGI 2.0.25.1 (64bit) on [Thu Nov  7 14:08:29 2024] ***
2024-11-07 16:08:29.305510+02:00compiled with version: 13.2.1 20240309 on 17 May 2024 06:11:37
2024-11-07 16:08:29.305517+02:00os: Linux-6.6.32-production+truenas #1 SMP PREEMPT_DYNAMIC Wed Oct  9 19:36:03 UTC 2024
2024-11-07 16:08:29.305526+02:00nodename: searxng-67fccb4498-h9rqd
2024-11-07 16:08:29.305532+02:00machine: x86_64
2024-11-07 16:08:29.305537+02:00clock source: unix
2024-11-07 16:08:29.305551+02:00pcre jit disabled
2024-11-07 16:08:29.305556+02:00detected number of CPU cores: 4
2024-11-07 16:08:29.305561+02:00current working directory: /usr/local/searxng
2024-11-07 16:08:29.305566+02:00detected binary path: /usr/sbin/uwsgi
2024-11-07 16:08:29.305610+02:00setgid() to 977
2024-11-07 16:08:29.305620+02:00setuid() to 977
2024-11-07 16:08:29.305625+02:00chdir() to /usr/local/searxng/searx/
2024-11-07 16:08:29.305635+02:00your memory page size is 4096 bytes
2024-11-07 16:08:29.305640+02:00detected max file descriptor number: 1048576
2024-11-07 16:08:29.310905+02:00building mime-types dictionary from file /etc/mime.types...1390 entry found
2024-11-07 16:08:29.310945+02:00lock engine: pthread robust mutexes
2024-11-07 16:08:29.310952+02:00thunder lock: disabled (you can enable it with --thunder-lock)
2024-11-07 16:08:29.310968+02:00uwsgi socket 0 bound to TCP address 0.0.0.0:30053 fd 3
2024-11-07 16:08:29.310974+02:00Python version: 3.12.7 (main, Oct  7 2024, 11:30:19) [GCC 13.2.1 20240309]
2024-11-07 16:08:29.329864+02:00Python main interpreter initialized at 0x7fd4faa6e950
2024-11-07 16:08:29.329900+02:00python threads support enabled
2024-11-07 16:08:29.329907+02:00your server socket listen backlog is limited to 100 connections
2024-11-07 16:08:29.329921+02:00your mercy for graceful operations on workers is 60 seconds
2024-11-07 16:08:29.329927+02:00mapped 603360 bytes (589 KB) for 16 cores
2024-11-07 16:08:29.329932+02:00*** Operational MODE: preforking+threaded ***
2024-11-07 16:08:29.332308+02:00added /usr/local/searxng/ to pythonpath.
2024-11-07 16:08:29.332343+02:00spawned uWSGI master process (pid: 7)
2024-11-07 16:08:29.332350+02:00spawned uWSGI worker 1 (pid: 10, cores: 4)
2024-11-07 16:08:29.332364+02:00spawned uWSGI worker 2 (pid: 12, cores: 4)
2024-11-07 16:08:29.332369+02:00spawned 4 offload threads for uWSGI worker 1
2024-11-07 16:08:29.332374+02:00spawned uWSGI worker 3 (pid: 16, cores: 4)
2024-11-07 16:08:29.332379+02:00spawned uWSGI worker 4 (pid: 19, cores: 4)
2024-11-07 16:08:29.332384+02:00spawned 4 offload threads for uWSGI worker 3
2024-11-07 16:08:29.333260+02:00spawned 4 offload threads for uWSGI worker 2
2024-11-07 16:08:29.333459+02:00spawned 4 offload threads for uWSGI worker 4
2024-11-07 16:08:29.598092+02:00Traceback (most recent call last):
2024-11-07 16:08:29.599078+02:00File "/usr/local/searxng/searx/__init__.py", line 24, in <module>
2024-11-07 16:08:29.599095+02:00settings, settings_load_message = searx.settings_loader.load_settings()
2024-11-07 16:08:29.599100+02:00^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-11-07 16:08:29.599117+02:00File "/usr/local/searxng/searx/settings_loader.py", line 194, in load_settings
2024-11-07 16:08:29.599123+02:00cfg_folder = get_user_cfg_folder()
2024-11-07 16:08:29.599147+02:00^^^^^^^^^^^^^^^^^^^^^
2024-11-07 16:08:29.599154+02:00File "/usr/local/searxng/searx/settings_loader.py", line 100, in get_user_cfg_folder
2024-11-07 16:08:29.599163+02:00if settings_path.is_dir():
2024-11-07 16:08:29.599169+02:00^^^^^^^^^^^^^^^^^^^^^^
2024-11-07 16:08:29.599174+02:00File "/usr/lib/python3.12/pathlib.py", line 875, in is_dir
2024-11-07 16:08:29.599179+02:00return S_ISDIR(self.stat().st_mode)
2024-11-07 16:08:29.599263+02:00^^^^^^^^^^^
2024-11-07 16:08:29.599332+02:00File "/usr/lib/python3.12/pathlib.py", line 840, in stat
2024-11-07 16:08:29.599729+02:00return os.stat(self, follow_symlinks=follow_symlinks)
2024-11-07 16:08:29.599851+02:00^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-11-07 16:08:29.599923+02:00PermissionError: [Errno 13] Permission denied: '/etc/searxng/settings.yml'
2024-11-07 16:08:29.599978+02:00unable to load app 0 (mountpoint='') (callable not found or import error)
2024-11-07 16:08:29.600044+02:00*** no app loaded. going in full dynamic mode ***
2024-11-07 16:08:29.620608+02:00Traceback (most recent call last):
2024-11-07 16:08:29.620643+02:00File "/usr/local/searxng/searx/__init__.py", line 24, in <module>
2024-11-07 16:08:29.620653+02:00settings, settings_load_message = searx.settings_loader.load_settings()
2024-11-07 16:08:29.620658+02:00^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-11-07 16:08:29.620663+02:00File "/usr/local/searxng/searx/settings_loader.py", line 194, in load_settings
2024-11-07 16:08:29.620681+02:00cfg_folder = get_user_cfg_folder()
2024-11-07 16:08:29.620687+02:00^^^^^^^^^^^^^^^^^^^^^
2024-11-07 16:08:29.620692+02:00File "/usr/local/searxng/searx/settings_loader.py", line 100, in get_user_cfg_folder
2024-11-07 16:08:29.620698+02:00if settings_path.is_dir():
2024-11-07 16:08:29.620703+02:00^^^^^^^^^^^^^^^^^^^^^^
2024-11-07 16:08:29.620712+02:00File "/usr/lib/python3.12/pathlib.py", line 875, in is_dir
2024-11-07 16:08:29.620717+02:00return S_ISDIR(self.stat().st_mode)
2024-11-07 16:08:29.620722+02:00^^^^^^^^^^^
2024-11-07 16:08:29.620727+02:00File "/usr/lib/python3.12/pathlib.py", line 840, in stat
2024-11-07 16:08:29.620732+02:00return os.stat(self, follow_symlinks=follow_symlinks)
2024-11-07 16:08:29.620737+02:00^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-11-07 16:08:29.620746+02:00PermissionError: [Errno 13] Permission denied: '/etc/searxng/settings.yml'
2024-11-07 16:08:29.620752+02:00unable to load app 0 (mountpoint='') (callable not found or import error)
2024-11-07 16:08:29.620757+02:00*** no app loaded. going in full dynamic mode ***
2024-11-07 16:08:29.645090+02:00Traceback (most recent call last):
2024-11-07 16:08:29.645183+02:00File "/usr/local/searxng/searx/__init__.py", line 24, in <module>
2024-11-07 16:08:29.645400+02:00settings, settings_load_message = searx.settings_loader.load_settings()
2024-11-07 16:08:29.645561+02:00^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-11-07 16:08:29.645666+02:00File "/usr/local/searxng/searx/settings_loader.py", line 194, in load_settings
2024-11-07 16:08:29.645898+02:00cfg_folder = get_user_cfg_folder()
2024-11-07 16:08:29.646023+02:00^^^^^^^^^^^^^^^^^^^^^
2024-11-07 16:08:29.646070+02:00File "/usr/local/searxng/searx/settings_loader.py", line 100, in get_user_cfg_folder
2024-11-07 16:08:29.646269+02:00if settings_path.is_dir():
2024-11-07 16:08:29.646378+02:00^^^^^^^^^^^^^^^^^^^^^^
2024-11-07 16:08:29.646443+02:00File "/usr/lib/python3.12/pathlib.py", line 875, in is_dir
2024-11-07 16:08:29.646823+02:00return S_ISDIR(self.stat().st_mode)
2024-11-07 16:08:29.646940+02:00^^^^^^^^^^^
2024-11-07 16:08:29.646987+02:00File "/usr/lib/python3.12/pathlib.py", line 840, in stat
2024-11-07 16:08:29.647353+02:00return os.stat(self, follow_symlinks=follow_symlinks)
2024-11-07 16:08:29.647478+02:00^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-11-07 16:08:29.647697+02:00PermissionError: [Errno 13] Permission denied: '/etc/searxng/settings.yml'
2024-11-07 16:08:29.647717+02:00unable to load app 0 (mountpoint='') (callable not found or import error)
2024-11-07 16:08:29.647723+02:00*** no app loaded. going in full dynamic mode ***
2024-11-07 16:08:29.649638+02:00Traceback (most recent call last):
2024-11-07 16:08:29.649758+02:00File "/usr/local/searxng/searx/__init__.py", line 24, in <module>
2024-11-07 16:08:29.649963+02:00settings, settings_load_message = searx.settings_loader.load_settings()
2024-11-07 16:08:29.650108+02:00^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-11-07 16:08:29.650181+02:00File "/usr/local/searxng/searx/settings_loader.py", line 194, in load_settings
2024-11-07 16:08:29.650402+02:00cfg_folder = get_user_cfg_folder()
2024-11-07 16:08:29.650518+02:00^^^^^^^^^^^^^^^^^^^^^
2024-11-07 16:08:29.650581+02:00File "/usr/local/searxng/searx/settings_loader.py", line 100, in get_user_cfg_folder
2024-11-07 16:08:29.650791+02:00if settings_path.is_dir():
2024-11-07 16:08:29.650904+02:00^^^^^^^^^^^^^^^^^^^^^^
2024-11-07 16:08:29.650951+02:00File "/usr/lib/python3.12/pathlib.py", line 875, in is_dir
2024-11-07 16:08:29.651448+02:00return S_ISDIR(self.stat().st_mode)
2024-11-07 16:08:29.651562+02:00^^^^^^^^^^^
2024-11-07 16:08:29.651642+02:00File "/usr/lib/python3.12/pathlib.py", line 840, in stat
2024-11-07 16:08:29.652030+02:00return os.stat(self, follow_symlinks=follow_symlinks)
2024-11-07 16:08:29.652196+02:00^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-11-07 16:08:29.652289+02:00PermissionError: [Errno 13] Permission denied: '/etc/searxng/settings.yml'
2024-11-07 16:08:29.652359+02:00unable to load app 0 (mountpoint='') (callable not found or import error)
2024-11-07 16:08:29.652441+02:00*** no app loaded. going in full dynamic mode ***
2024-11-07 16:08:41.327042+02:00--- no python application found, check your startup logs for errors ---
2024-11-07 16:08:41.327146+02:00[pid: 16|app: -1|req: -1/1] 172.16.0.1 () {26 vars in 314 bytes} [Thu Nov  7 14:08:41 2024] GET /healthz => generated 21 bytes in 0 msecs (HTTP/1.1 500) 3 headers in 102 bytes (0 switches on core 0)
2024-11-07 16:08:46.327207+02:00--- no python application found, check your startup logs for errors ---
2024-11-07 16:08:46.327263+02:00[pid: 19|app: -1|req: -1/2] 172.16.0.1 () {26 vars in 314 bytes} [Thu Nov  7 14:08:46 2024] GET /healthz => generated 21 bytes in 0 msecs (HTTP/1.1 500) 3 headers in 102 bytes (0 switches on core 0)
2024-11-07 16:08:51.327106+02:00--- no python application found, check your startup logs for errors ---
2024-11-07 16:08:51.327176+02:00[pid: 10|app: -1|req: -1/3] 172.16.0.1 () {26 vars in 314 bytes} [Thu Nov  7 14:08:51 2024] GET /healthz => generated 21 bytes in 0 msecs (HTTP/1.1 500) 3 headers in 102 bytes (0 switches on core 2)

and the last two entries just keep repeating every second or so

I added the SECRET_KEY env variable (random string) on app settings and no python application found was gone and the log just ended with this:

2024-11-07 16:13:45.561248+02:00Traceback (most recent call last):
2024-11-07 16:13:45.561268+02:00File "/usr/local/searxng/searx/__init__.py", line 24, in <module>
2024-11-07 16:13:45.561352+02:00settings, settings_load_message = searx.settings_loader.load_settings()
2024-11-07 16:13:45.561810+02:00^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-11-07 16:13:45.561826+02:00File "/usr/local/searxng/searx/settings_loader.py", line 194, in load_settings
2024-11-07 16:13:45.561831+02:00cfg_folder = get_user_cfg_folder()
2024-11-07 16:13:45.561837+02:00^^^^^^^^^^^^^^^^^^^^^
2024-11-07 16:13:45.561842+02:00File "/usr/local/searxng/searx/settings_loader.py", line 100, in get_user_cfg_folder
2024-11-07 16:13:45.561855+02:00if settings_path.is_dir():
2024-11-07 16:13:45.561862+02:00^^^^^^^^^^^^^^^^^^^^^^
2024-11-07 16:13:45.561867+02:00File "/usr/lib/python3.12/pathlib.py", line 875, in is_dir
2024-11-07 16:13:45.562210+02:00return S_ISDIR(self.stat().st_mode)
2024-11-07 16:13:45.562224+02:00^^^^^^^^^^^
2024-11-07 16:13:45.562239+02:00File "/usr/lib/python3.12/pathlib.py", line 840, in stat
2024-11-07 16:13:45.562736+02:00return os.stat(self, follow_symlinks=follow_symlinks)
2024-11-07 16:13:45.562756+02:00^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-11-07 16:13:45.562762+02:00PermissionError: [Errno 13] Permission denied: '/etc/searxng/settings.yml'
2024-11-07 16:13:45.562767+02:00unable to load app 0 (mountpoint='') (callable not found or import error)
2024-11-07 16:13:45.562781+02:00*** no app loaded. going in full dynamic mode ***
2024-11-07 16:13:49.445475+02:00[FATAL tini (1)] Unexpected error when forwarding signal: 'Operation not permitted'

So I noticed

Permission denied: '/etc/searxng/settings.yml'

Apparently the files were created and they are not empty. Both settings.yml and uwsgi.ini have root:568 (568 is apps GID/UID on TrueNAS). Tried chown 568:568 both, and python app error came back. Tried fully recreating the app with SECRET_KEY this time, and python error again was there.

Should I get back to TrueNAS Charts repo to ask for help about permissions? It's way over my head I think

dalf Nov 7, 2024

Should I get back to TrueNAS Charts repo to ask for help about permissions?

Yes, I think so looking at your analysis

Karmalakas Nov 8, 2024
Author

OK, so it seems it might've been my inexperience. Here's a copy of my comment on charts repo:

So I decided to try removing everything related to SearXNG and recreate from scratch. While creating a dataset, this time instead of choosing "Apps" preset, I left it on "Generic" - and it seems it worked. Everything loads and search itself runs, although in the logs I see this four times:

2024-11-08 16:38:48.319623+00:002024-11-08 16:38:48,319 WARNING:searx.botdetection.config: missing config file: /etc/searxng/limiter.toml

Anyway, considering this as resolved, but if you have any insight about this limiter.toml, I'd be grateful to hear it 🙏

Answer selected by Karmalakas

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Python app missing when deploying in TrueNAS Scale #3998

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 1 comment 3 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Python app missing when deploying in TrueNAS Scale #3998

Uh oh!

Uh oh!

Karmalakas Nov 6, 2024

Replies: 1 comment · 3 replies

Uh oh!

Uh oh!

dalf Nov 7, 2024

Uh oh!

Uh oh!

Karmalakas Nov 7, 2024 Author

Uh oh!

dalf Nov 7, 2024

Uh oh!

Karmalakas Nov 8, 2024 Author

Karmalakas
Nov 6, 2024

Replies: 1 comment 3 replies

dalf
Nov 7, 2024

Karmalakas Nov 7, 2024
Author

Karmalakas Nov 8, 2024
Author