-
Notifications
You must be signed in to change notification settings - Fork 11
Description
Is your feature request related to a problem? Please describe.
Let's say I want to collect logs from a crowded CloudTrail AWS Organization (hundred of accounts, loads of Gb per day), because I want to troubleshoot an issue that happened TODAY.
But Tailpipe CLI doesn't allow me to collect only today's logs. it want --from T-1d as a minimum, which means yesterday logs too, so more than 3x (on average) the log volume I need.
I still tried to collect at the end of my working day, but collection crashed while I wasn't on my workstation anymore.
Fast forward 2 days later: I still want those logs of this precise day. But now it's worse, as Tailpipe forces me to collect more days of logs to get the logs of the day I'm interested in.
Describe the solution you'd like
- I'd like a
tailpipe collect aws_cloudtrail_log.trail --from T-<whatever>Hor at least a--from T-0davailable. - Some matching
--to <...>syntax to avoid being forced to collect logs until "now". With this, I'll be able to specify a precise timeframe of logs to collect (from->towindow). [edit: could be useful to purge old data too withtailpipe partition delete]
Describe alternatives you've considered
❯ tailpipe collect aws_cloudtrail_log.trail --from T-12H
Error: 'from' time must be at least 1 day in the past
Why this limitation?
Additional context
None I can think of. Tell me if more details would be useful.
Regards, and good luck getting this tool in a good shape, I love what you are trying to do, with the PipeVerse ;)