-
Notifications
You must be signed in to change notification settings - Fork 50
Description
Hi,
I am using google-fluentd version 1.11.2. I've noticed that google-fluentd silently ignores log entries with improper time field. Here is configuration:
<source>
@type tail
format json
path /home/jenkins/workspace/*/storage/logs/*.log
pos_file /var/lib/google-fluentd/pos/jenkins.pos
read_from_head true
tag jobs-logs
</source>
<filter jobs-logs*>
@type record_transformer
<record>
severity ${record["severity"]}
service "jobs"
</record>
</filter>
Sample log entry looks like that:
{
"log_name": "big_query",
"message": "new request",
"message_template": "new request",
"time": "2020-11-23T18:57:08.694567+00:00",
"severity": "INFO",
"stack": {
"thread": 140232798734080,
"process": 6,
"exc_info": null,
"exc_text": null,
"stack_info": null
},
"extra": {
"req": {
"query": "...",
"project": "xxx",
"params": "xxx"
},
"res": {
"state": "DONE",
"duration": 1.209,
"latency": 1.552014112,
"tb_billed": 1.0486e-05,
"cost_uds": 5,
"cache_hit": false,
"errors": null
},
"extra_data": {
"job_name": "xxx"
}
}
}
The problem here is probably with the time
field. By default google-fluentd expects a float. However, these log entries are not visible in LogsViewer and there is no information in logs or in a verbose mode that such log entries are dropped.
When I add to the configuration that time_type string
and also some (invalid for test case) time_format %Y-%-m...
, then google-fluentd correctly reports that time_format is invalid.
I think google-fluentd should also report about the first situation. Without any information, it made debugging missing logs very difficult.