这是indexloc提供的服务,不要输入任何密码
Skip to content

Logs are too verbose and generate high amount of data #44

@christian-roggia

Description

@christian-roggia

First of all, I would like to thank you for the amazing work you did here, outstanding!

This ticket is related to issues we encountered in a cloud environment, and more specifically in Kubernetes deployments.

A little background about our setup: we are migrating around 10 million records from ETCD to CETE for a total of ca. 30 GB of data. The data migrated contains JSON, HTML, and other formats.

I would like to address the issue of logs as currently too much traffic/data is being generated and logs are too verbose:

  • Readiness / Liveness check should not log anything unless the log level is set to DEBUG or anything similar. (NOTE: In a kubernetes environment checks are executed regularly every 1-5 seconds).
  • Set / Get / Delete / etc. operations should not generate a log for each change executed - that is a DEBUG log, to give some context when we launched the first migration we have seen over 50 MB of logs generated per second. We have been forced to set the log level to WARNING to avoid high costs for logs ingestion by our cloud provider, which is not ideal.
  • NOT FOUND errors should be less verbose (there are 3 logs generated for each failed lookup, one of which contains the stacktrace). I am not even sure whether a NOT FOUND error should be logged at all - especially with level ERROR - as it is the expected behavior and the server did not encounter any error, the resource was simply missing.

I will update this ticket with more information about other logs as we encounter them.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions