这是indexloc提供的服务,不要输入任何密码
Skip to content

Collection timeframe enhancement #394

@aviv-florentdutheil

Description

@aviv-florentdutheil

Is your feature request related to a problem? Please describe.
Let's say I want to collect logs from a crowded CloudTrail AWS Organization (hundred of accounts, loads of Gb per day), because I want to troubleshoot an issue that happened TODAY.
But Tailpipe CLI doesn't allow me to collect only today's logs. it want --from T-1d as a minimum, which means yesterday logs too, so more than 3x (on average) the log volume I need.

I still tried to collect at the end of my working day, but collection crashed while I wasn't on my workstation anymore.
Fast forward 2 days later: I still want those logs of this precise day. But now it's worse, as Tailpipe forces me to collect more days of logs to get the logs of the day I'm interested in.

Describe the solution you'd like

  1. I'd like a tailpipe collect aws_cloudtrail_log.trail --from T-<whatever>H or at least a --from T-0d available.
  2. Some matching --to <...> syntax to avoid being forced to collect logs until "now". With this, I'll be able to specify a precise timeframe of logs to collect (from -> to window). [edit: could be useful to purge old data too with tailpipe partition delete]

Describe alternatives you've considered

❯ tailpipe collect aws_cloudtrail_log.trail --from T-12H
Error: 'from' time must be at least 1 day in the past

Why this limitation?

Additional context
None I can think of. Tell me if more details would be useful.

Regards, and good luck getting this tool in a good shape, I love what you are trying to do, with the PipeVerse ;)

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions