Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiple Conditions with AND/OR Logic for ISM Transitions #1261

Open
basht0p opened this issue Sep 18, 2024 · 0 comments
Open

Multiple Conditions with AND/OR Logic for ISM Transitions #1261

basht0p opened this issue Sep 18, 2024 · 0 comments
Labels

Comments

@basht0p
Copy link

basht0p commented Sep 18, 2024

Is your feature request related to a problem?
I mean, to me it's a problem, but no.

What solution would you like?
It would be awesome to be able to apply some AND/OR logic to the conditions for transitions. For example:

"transitions": [
    {
        "state_name": "cold",
        "condition_logic": "and",
        "conditions": {
            "min_index_age": "90d",
            "min_rollover_age": "7d"
        }
    },
    {
        "state_name": "cold",
        "condition_logic": "or",
        "conditions": {
            "min_size": "20gb",
            "min_doc_count": "10000000"
        }
    }
]

In pseudo-query language, this would effectively be the same as:
(min_index_age: 90d AND min_rollover_age: 7d) OR (min_size: 20gb OR min_doc_count: 10000000)

I realize the OR/OR is redundant, as (from what I understand) the ISM already does a sequential "hunt" through the conditions looking for a first match, and I suppose the only novel idea here is adding the AND logic.

My use case, for example, is that I have some index sets which generate up to 40GB of documents per day, and I have others which generate as low as 40GB per quarter. I would like to keep my rotation policy based on size, but I don't want to move an active write index to a cold state, even if it's exceeded 90 days. If it's 101 days old, but is still the active write index, the transition should not apply (hence the additional min_rollover_age).

What alternatives have you considered?
I'm certain the tools exist within OpenSearch to allow me to "tier" my index sets based on their ingress volume/rate/etc, and apply different policies based on that, but that solution feels a bit clunky and I feel as if applying the logic to allow multiple conditions would be a simpler solution overall.

Do you have any additional context?
I am currently on a 5 node cluster running 2.15, however it does not look like this feature exists in later versions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant