-
Notifications
You must be signed in to change notification settings - Fork 46
Description
According to the API specification, when creating a job, we must provide the bucket name and the prefix of the files with aggregatable reports to be included in the aggregation. Since I want to perform the aggregation every hour, it seems necessary to have a separate prefix for each hour. For example:
- /data/2024-07-19/00/...
- /data/2024-07-19/01/...
- /data/2024-07-19/02/...
- /data/2024-07-19/03/...
- etc.
However, if I need to perform an aggregation over a 6-hour interval (using different filtering id), I encounter a problem. The API only allows one prefix, which means I would need to copy the data to a new location. This approach seems impractical and inefficient.
It would be highly beneficial if the aggregation service could accept a list of prefixes. This change would allow more flexibility in specifying the data intervals for aggregation without needing to duplicate data.