-
Notifications
You must be signed in to change notification settings - Fork 28
Open
Description
First of all, I would like to thank you for the amazing work you did here, outstanding!
This ticket is related to issues we encountered in a cloud environment, and more specifically in Kubernetes deployments.
A little background about our setup: we are migrating around 10 million records from ETCD to CETE for a total of ca. 30 GB of data. The data migrated contains JSON, HTML, and other formats.
I would like to address the issue of logs as currently too much traffic/data is being generated and logs are too verbose:
- Readiness / Liveness check should not log anything unless the log level is set to DEBUG or anything similar. (NOTE: In a kubernetes environment checks are executed regularly every 1-5 seconds).
- Set / Get / Delete / etc. operations should not generate a log for each change executed - that is a DEBUG log, to give some context when we launched the first migration we have seen over 50 MB of logs generated per second. We have been forced to set the log level to WARNING to avoid high costs for logs ingestion by our cloud provider, which is not ideal.
- NOT FOUND errors should be less verbose (there are 3 logs generated for each failed lookup, one of which contains the stacktrace). I am not even sure whether a NOT FOUND error should be logged at all - especially with level ERROR - as it is the expected behavior and the server did not encounter any error, the resource was simply missing.
I will update this ticket with more information about other logs as we encounter them.
Metadata
Metadata
Assignees
Labels
No labels