这是indexloc提供的服务,不要输入任何密码
Skip to content

Authelia stops responding on /api/authz/forward-auth #9936

@alteriks

Description

@alteriks

Version

v4.39.0

Deployment Method

Docker

Reverse Proxy

Traefik

Reverse Proxy Version

3.2.1

Description

After couple of weeks authelia randomly stops responding on /api/authz/forward-auth endpoint.
Any user that tries to login using authelia receives 500 internal server error from traefik proxy.
I've attached debug log from Authelia it's configuration as well as docker-compose.yml.

As soon as authelia container is restarted users can login once again.

Reproduction

This behavior is erratic. It happened couple of times. I've enabled trace logs to gather more information

Expectations

No response

Configuration (Authelia)

etc/authelia/configuration.yml:

server:
  address: 'tcp://:9091'

log:
  level: 'debug'

totp:
  issuer: 'example.com'

identity_validation:
  reset_password:
    jwt_secret: '__SECRET__'

ntp:
  address: 'udp://pool.ntp.example.com:123'
  version: 3
  max_desync: '3s'

session:
  # This secret can also be set using the env variables AUTHELIA_SESSION_SECRET_FILE
  secret: '__SECRET__'

  cookies:
    - name: 'authelia_session'
      domain: 'example.com' # Should match whatever your root protected domain is
      authelia_url: 'https://authelia.example.com'
      expiration: '1 hour'
      inactivity: '5 minutes'

  redis:
    host: '127.0.0.1'
    port: 6379
    # This secret can also be set using the env variables AUTHELIA_SESSION_REDIS_PASSWORD_FILE
    # password: authelia

regulation:
  max_retries: 3
  find_time: '2 minutes'
  ban_time: '15 minutes'

storage:
  encryption_key: 'dkmL5iqwPsom5RLVLdMCos5cd2rF4uSJqFweiTqPyNDaD5KAv5HnMpkqHYE7wFcL7'
  local:
    path: '/authelia/db.sqlite3'

# Used for adding 2FA (OTP/WebAuthn)
notifier:
  smtp:
    username: 'mailman'
    # This secret can also be set using the env variables AUTHELIA_NOTIFIER_SMTP_PASSWORD_FILE
    password: '__SECRET__'
    address: 'submissions://smtp.garage.example.com:465'
    sender: 'authelia@example.com'
    # mailrise/apprise:Failed to send Telegram notification to 414546848: Bad Request: message is too long
    disable_html_emails: true

authentication_backend:
  password_reset:
    disable: false
  # How often authelia should check if there is a user update in LDAP
  refresh_interval: 1m
  ldap:
    address: 'ldaps://ldap.example.com'
    implementation: 'lldap'
    timeout: '5s'
    start_tls: false
    tls:
      skip_verify: false
      minimum_version: 'TLS1.2'
      maximum_version: 'TLS1.3'
    pooling:
      enable: true
      count: 5
      retries: 2
      timeout: '10 seconds'
    base_dn: 'DC=example,DC=com'
    user: 'uid=authelia,OU=people,DC=example,DC=com'
    password: '__SECRET__'
    additional_users_dn: 'OU=people'
    users_filter: '(&({username_attribute}={input})(objectClass=person))'
    additional_groups_dn: 'OU=groups'
    group_search_mode: 'filter'
    groups_filter: '(&(member={dn})(objectclass=groupOfUniqueNames))'
    # permit_referrals: false

password_policy:
  standard:
    enabled: true
    min_length: 64
    max_length: 0
    require_uppercase: true
    require_lowercase: true
    require_number: true
    require_special: true

webauthn:
  disable: false
  enable_passkey_login: true
  display_name: Authelia
  attestation_conveyance_preference: indirect
  selection_criteria:
    user_verification: preferred


etc/authelia/configuration.acl.yml:

###############################################################
#                   Authelia ACL                              #
###############################################################

access_control:
  networks:
    - name: 'watch.example.com'
      networks:
        - '10.5.10.20/32'
    - name: 'arr.example.com'
      networks:
        - '10.5.10.18'
  rules:
    # Disable authelia for watch.example.com
    - domain: '*.example.com'
      networks:
        - 'watch.example.com'
      policy: 'bypass'

    - domain:
        - sonarr.example.com
        - radarr.example.com
      resources:
        - '^/api([/?].*)?$'
      networks:
        - 'arr.example.com' # this probably doesn't work... see above
      policy: bypass

    - domain:
        - 'radarr.example.com'
        - 'sonarr.example.com'
        - 'nzbget.example.com'
      subject:
        - 'group:arr'
      policy: 'one_factor'

    - domain:
        - 'ara.example.com'
      subject:
        - 'group:superadmin'
      policy: 'one_factor'

    - domain:
        - 'changedetection.example.com'
      subject:
        - 'group:changedetection'
      policy: 'one_factor'

    - domain_regex: '^traefik\.\w+\.example\.com$'
      subject:
        - 'group:superadmin'
      policy: 'one_factor'

    - domain_regex: '^syncthing\.\w+\.example\.com$'
      subject:
        - 'group:superadmin'
      policy: 'one_factor'

    - domain: 'authelia.example.com'
      policy: 'one_factor'

    - domain: 'secure.example.com'
      policy: 'two_factor'


etc/authelia/oidc.yml:

identity_providers:
  oidc:
    jwks:
      - key_id: 'example'
        algorithm: 'RS256'
        use: 'sig'
        key: |
          -----BEGIN PRIVATE KEY-----
          -----END PRIVATE KEY-----
    ## The other portions of the mandatory OpenID Connect 1.0 configuration go here.
    claims_policies:
      ## Creates the 'default' claims policy.
      default:
        id_token: ['groups', 'email', 'email_verified', 'alt_emails', 'preferred_username', 'name']
    ## See: https://www.authelia.com/c/oidc
    clients:
      - client_id: 'gitea'
        client_name: 'Gitea'
        client_secret: '__SECRET__'
        public: false
        authorization_policy: 'two_factor'
        redirect_uris:
          - 'https://gitea.example.com/user/oauth2/authelia/callback'
        scopes:
          - 'openid'
          - 'email'
          - 'profile'
        userinfo_signed_response_alg: 'none'
        token_endpoint_auth_method: 'client_secret_basic'
        consent_mode: implicit # disables consent requests

      - client_id: 'grafana'
        client_name: 'Grafana'
        client_secret: '__SECRET__'
        public: false
        authorization_policy: 'two_factor'
        require_pkce: true
        pkce_challenge_method: 'S256'
        redirect_uris:
          - 'https://grafana.example.com/login/generic_oauth'
        scopes:
          - 'openid'
          - 'profile'
          - 'groups'
          - 'email'
        userinfo_signed_response_alg: 'none'
        token_endpoint_auth_method: 'client_secret_basic'
        consent_mode: implicit # disables consent requests

      - client_id: 'harbor'
        client_name: 'Harbor'
        client_secret: '__SECRET__'
        public: false
        authorization_policy: 'two_factor'
        redirect_uris:
          - 'https://harbor.example.com/c/oidc/callback'
        grant_types:
          - 'refresh_token'
          - 'authorization_code'
        scopes:
          - 'openid'
          - 'profile'
          - 'groups'
          - 'email'
          - 'offline_access'
        userinfo_signed_response_alg: 'none'
        consent_mode: implicit # disables consent requests

      - client_id: 'headscale'
        client_name: 'headscale'
        client_secret: '__SECRET__'
        public: false
        claims_policy: 'default'
        authorization_policy: 'two_factor'
        require_pkce: true
        pkce_challenge_method: 'S256'
        redirect_uris:
          - 'https://headscale.example.com/oidc/callback'
        scopes:
          - 'openid'
          - 'profile'
          - 'groups'
          - 'email'
        userinfo_signed_response_alg: 'none'
        token_endpoint_auth_method: 'client_secret_basic'
        consent_mode: implicit # disables consent requests

      - client_id: 'immich'
        client_name: 'immich'
        client_secret: '__SECRET__'
        public: false
        authorization_policy: 'two_factor'
        redirect_uris:
          - 'https://immich.example.com/auth/login'
          - 'https://immich.example.com/user-settings'
          - 'app.immich:///oauth-callback'
        scopes:
          - 'openid'
          - 'profile'
          - 'email'
        userinfo_signed_response_alg: 'none'
        token_endpoint_auth_method: "client_secret_post" # https://github.com/immich-app/immich/releases/tag/v1.132.3
        # TODO: claims STORAGE LABEL CLAIM=preferred_username, STORAGE QUOTA CLAIM=immich_quota
        consent_mode: implicit # disables consent requests

      - client_id: 'paperless'
        client_name: 'Paperless'
        client_secret: '__SECRET__'
        public: false
        authorization_policy: 'two_factor'
        require_pkce: true
        pkce_challenge_method: 'S256'
        redirect_uris:
          - 'https://paperless.example.com/accounts/oidc/authelia/login/callback/'
        scopes:
          - 'openid'
          - 'profile'
          - 'email'
          - 'groups'
        userinfo_signed_response_alg: 'none'
        token_endpoint_auth_method: 'client_secret_basic'
        consent_mode: implicit # disables consent requests

      - client_id: 'pbs'
        client_name: 'Proxmox Backup Server'
        client_secret: '__SECRET__'
        public: false
        authorization_policy: 'two_factor'
        require_pkce: true
        pkce_challenge_method: 'S256'
        redirect_uris:
          - 'https://pbs.example.com'
        scopes:
          - 'openid'
          - 'profile'
          - 'email'
        userinfo_signed_response_alg: 'none'
        consent_mode: implicit # disables consent requests

      - client_id: 'proxmox'
        client_name: 'Proxmox'
        client_secret: '__SECRET__'
        public: false
        authorization_policy: 'two_factor'
        require_pkce: true
        pkce_challenge_method: 'S256'
        redirect_uris:
          - 'https://proxmox.example.com'
          - 'https://t800.example.com:8006'
          - 'https://hal.example.com:8006'
        scopes:
          - 'openid'
          - 'profile'
          - 'email'
        userinfo_signed_response_alg: 'none'
        consent_mode: implicit # disables consent requests

      - client_id: 'proxmox_homelab'
        client_name: 'Proxmox Homelab'
        client_secret: '__SECRET__'
        public: false
        authorization_policy: 'two_factor'
        require_pkce: true
        pkce_challenge_method: 'S256'
        redirect_uris:
          - 'https://homelab.example.com'
          - 'https://legion.example.com:8006'
          - 'https://skynet.example.com:8006'
          - 'https://mother.example.com:8006'
        scopes:
          - 'openid'
          - 'profile'
          - 'email'
        userinfo_signed_response_alg: 'none'
        consent_mode: implicit # disables consent requests

      - client_id: 'tandoor'
        client_name: 'Tandoor'
        client_secret: '__SECRET__'
        public: false
        authorization_policy: 'two_factor'
        require_pkce: true
        pkce_challenge_method: 'S256'
        redirect_uris:
          - 'https://tandoor.example.com/accounts/oidc/authelia/login/callback/'
        scopes:
          - 'openid'
          - 'profile'
          - 'email'
          - 'groups'
        userinfo_signed_response_alg: 'none'
        token_endpoint_auth_method: 'client_secret_basic'
        consent_mode: implicit # disables consent requests

      - client_id: 'vikunja'
        client_name: 'Vikunja'
        client_secret: '__SECRET__'
        public: false
        authorization_policy: 'two_factor'
        redirect_uris:
          - 'https://vikunja.example.com/auth/openid/authelia'
        scopes:
          - 'openid'
          - 'profile'
          - 'email'
        userinfo_signed_response_alg: 'none'
        token_endpoint_auth_method: 'client_secret_basic'
        consent_mode: implicit # disables consent requests



docker-compose.yml:

services:
  authelia:
    image: 'harbor.example.com/alteriks/authelia/authelia'
    container_name: 'authelia'
    volumes:
      - '/data/docker/authelia:/authelia' #db.sqlite3
      - './etc/authelia/configuration.yml:/config/configuration.yml'
      - './etc/authelia/configuration.acl.yml:/config/configuration.acl.yml'
      - './etc/authelia/oidc.yml:/config/oidc.yml'
    network_mode: host
    command:
      - 'authelia'
      - '--config=/config/configuration.yml'
      - '--config=/config/configuration.acl.yml'
      - '--config=/config/oidc.yml'
    labels:
      traefik.enable: 'true'
      traefik.http.routers.authelia.rule: 'Host(`authelia.example.com`)'
      traefik.http.routers.authelia.entrypoints: 'https'
      traefik.http.routers.authelia.tls: 'true'
      traefik.http.routers.authelia.tls.certresolver: 'le'
      traefik.http.routers.authelia.middlewares: 'hsts@docker'

      traefik.http.services.authelia.loadbalancer.server.port: 9091
    restart: 'unless-stopped'
    environment:
      TZ: 'Europe/Warsaw'
    deploy:
      restart_policy:
        condition: any
      resources:
        limits:
          memory: 300M
        reservations:
          memory: 50M

  redis:
    image: 'redis:alpine'
    container_name: 'redis'
    volumes:
      - '/data/docker/authelia/redis:/data'
    network_mode: host
    healthcheck:
      test: redis-cli ping || exit 1
    restart: 'unless-stopped'
    environment:
      TZ: 'Europe/Warsaw'
    deploy:
      resources:
        limits:
          memory: 500M
        reservations:
          memory: 50M

  traefik:
    image: 'harbor.example.com/alteriks/traefik'
    container_name: 'traefik'
    volumes:
      - /data/docker/traefik/etc/acme:/etc/acme
      - '/var/run/docker.sock:/var/run/docker.sock'
    network_mode: host
    healthcheck:
      test: ["CMD", "traefik", "healthcheck", "--entryPoints.ping.address=:8081", "--ping.entrypoint=ping", "--ping"]
    labels:
      traefik.enable: 'true'
      traefik.http.routers.api.rule: 'Host(`traefik.$HOSTNAME.example.com`)'
      traefik.http.routers.api.entrypoints: 'https'
      traefik.http.routers.api.service: 'api@internal'
      traefik.http.routers.api.tls: 'true'
      traefik.http.routers.api.tls.certresolver: 'le'
      traefik.http.routers.api.middlewares: 'authelia@docker,hsts@docker'
      traefik.http.services.traefik.loadbalancer.server.port: 8081

      traefik.http.routers.ping.service: ping@internal
      traefik.http.routers.ping.entrypoints: https
      traefik.http.routers.ping.rule: Host(`traefik.$HOSTNAME.example.com`) && PathPrefix(`/ping`)
      traefik.http.routers.ping.tls: true

      # HSTS
      traefik.http.middlewares.hsts.headers.stsSeconds: 31536000
      traefik.http.middlewares.hsts.headers.forceSTSHeader: true
      traefik.http.middlewares.hsts.headers.stsPreload: true
      traefik.http.middlewares.hsts.headers.stsIncludeSubdomains: true

      # fix: error="middleware \"authelia@docker\" does not exist", when traefik is started before authelia
      traefik.http.middlewares.authelia.forwardauth.address: 'http://127.0.0.1:9091/api/authz/forward-auth'
      traefik.http.middlewares.authelia.forwardauth.trustForwardHeader: 'true'
      traefik.http.middlewares.authelia.forwardauth.authResponseHeaders: 'Remote-User,Remote-Groups,Remote-Name,Remote-Email'

    command:
      - '--api'
      - "--ping"

      # Logging
      - '--log=true'
      - "--accesslog=false"
      - '--log.level=INFO'
      # - '--log.level=DEBUG'

      # disable BS
      - "--global.checknewversion=false"
      - "--global.sendanonymoususage=false"
      - "--api.disabledashboardad" # Disable ad in the dashboard.

      - '--providers.docker=true'
      - '--providers.docker.exposedByDefault=false'

      # Entrypoints
      - '--entrypoints.http=true'
      - '--entrypoints.http.address=:80'
      - '--entrypoints.http.http.redirections.entrypoint.to=https'
      - '--entrypoints.http.http.redirections.entrypoint.scheme=https'
      - '--entrypoints.https=true'
      - '--entrypoints.https.address=:443'
      - "--entrypoints.traefik.address=:8081" # Default :8080

      # Authelia Third Party Hosts
      # Without it other traefiks using authelia middleware will get
      # `400 Bad Request`, when requesting https://authelia.example.com/api/authz/forward-auth
      - "--entryPoints.https.forwardedHeaders.trustedIPs=10.5.10.0/24,10.5.2.0/27"
      - "--entryPoints.https.proxyProtocol.trustedIPs=10.5.10.0/24,10.5.2.0/27"

      # Lets Encrypt production
      - "--certificatesResolvers.le.acme.dnsChallenge=true"
      - "--certificatesResolvers.le.acme.dnsChallenge.provider=ovh"
      - "--certificatesResolvers.le.acme.email=alteriks@example.com"
      - "--certificatesResolvers.le.acme.storage=/etc/acme/acme_le.json"
      - "--certificatesresolvers.le.acme.caserver=https://acme-v02.api.letsencrypt.org/directory"

      # Lets Encrypt staging
      - "--certificatesResolvers.le_staging.acme.dnsChallenge=true"
      - "--certificatesResolvers.le_staging.acme.dnsChallenge.provider=ovh"
      - "--certificatesResolvers.le_staging.acme.email=alteriks@example.com"
      - "--certificatesResolvers.le_staging.acme.storage=/etc/acme/acme_le_staging.json"
      - "--certificatesresolvers.le_staging.acme.caserver=https://acme-staging-v02.api.letsencrypt.org/directory"

      # HSTS
      - "traefik.http.middlewares.hsts.headers.stsSeconds=31536000"
      - "traefik.http.middlewares.hsts.headers.forceSTSHeader=true"
      - "traefik.http.middlewares.hsts.headers.stsPreload=true"
      - "traefik.http.middlewares.hsts.headers.stsIncludeSubdomains=true"

    deploy:
      restart_policy:
        condition: any
      resources:
        limits:
          memory: 300M
        reservations:
          memory: 50M

Build Information

Last Tag: v4.39.0
State: tagged clean
Branch: v4.39.0
Commit: bbfa41712ce1b3e39b4798b18bd5f78e68470720
Build Number: 41111
Build OS: linux
Build Arch: amd64
Build Compiler: gc
Build Date: Sun, 16 Mar 2025 21:05:57 +1100
Extra: 

Go:
    Version: go1.24.1
    Module Path: github.com/authelia/authelia/v4
    Executable Path: github.com/authelia/authelia/v4/cmd/authelia

Logs (Authelia)

See first comment

Logs (Proxy / Application)

Documentation

No response

Generative AI

No

Pre-Submission Checklist

  • I agree to follow the Code of Conduct

  • This is a bug report and not a support request

  • I have read the security policy and this bug report is not a security issue or security related issue

  • I have either included the complete configuration file or I am sure it's unrelated to the configuration

  • I have either included the complete debug / trace logs or the output of the build-info command if the logs are not relevant

  • I have provided all of the required information in full with the only alteration being reasonable sanitization in accordance with the Troubleshooting Sanitization reference guide

  • I have checked for related proxy or application logs and included them if available

  • I have checked for related issues and checked the documentation

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions