Qualification tool: Filter based on timestamp in event logs

[nartal1]

User story:

1. As a user, I want to select at most k log files overall, keeping either the newest or oldest k logs (based on the timestamp inside the event log)
2. As a user, I want to select at most k log files for each unique application name, keeping either the newest or oldest k logs so that I can ensure that I process every distinct application I have run even if I process only a small subset of logs

[nartal1]

build

[pxLi]

build

[tgravescs]

I'm debating on the names here. I would think most people probably want to filter based on the app time so I wonder if we use the 100-newest to be app time and then make like a 100-newest-filesystem for filesystem time. I hate to change the config but don't think many people probably use it yet.

[nartal1]

Yes, I also don't think many people are using this config yet. I am fine changing if you think it makes more sense.
"100-newest-per-app-name" - per application based on app time.
"100-newest" - based on app time.
"100-newest-filesystem" - based on filesystem time.

[willb]

I agree that filtering based on filesystem metadata isn’t likely to be particularly useful (and find(1) will do that in any case, so it doesn’t necessarily need to be built into a tool).

In the prototype, we had --limit X (to choose the newest X logs) and --per-app-limit X (to choose at most the X newest logs per application name). We also had a --oldest flag to override the ordering. I thought this was a pretty sensible UX and in general prefer multiple options to configure multiple aspects of functionality instead of having a string to parse as an argument.

For time-based filtering, there are a couple of interesting cases:

• since a certain (human-readable) time
• between a range of human-readable times
• in the last k days (or hours / weeks / months)

I think that a reasonable command-line UX for these would look like this:

• --since 7/1/2021
• --since 7/1/2021 --before 7/15/2021
• --in-last 5d

[nartal1]

Thanks @willb .
We do have flag for : the last k days(mins/hours/weeks/months) - start-app-time . The input is not the date format though. It is : --start-app-time 2d (past 2days), 3w(past 3 weeks) and so on.

[nartal1]

build

[nartal1]

build

[tgravescs]

I think only minor thing would be missed the comment about changing the help/usage output to put the filesystem based options last since we think they will be used least

[tgravescs]

build