Auto-follow all indices with CCR except for system indices

[sachin-frayne]

I would like to be able to follow any new index created in my leader cluster except for system indices. I assumed *,-.* would work but I get a message pop up saying,

Index name examples
The above settings will generate index names that look like this:
2021-01-19,-.2021-01-19
2021-01-20,-.2021-01-20
2021-01-21,-.2021-01-21

This implies that the index name will have ,-. inside the actual index name. I have also tested this in 7.10.1 and can confirm that this does not in fact match any of my indices (e.g. a-index, 1-index, and so on). As a workaround, I can create an auto-follow pattern for every alphanumeric character, but this is not ideal

PUT /_ccr/auto_follow/a
{
  "remote_cluster": "<remote_cluster>",
  "leader_index_patterns": [
    "a*"
  ],
  "follow_index_pattern": "{{leader_index}}"
}

PUT /_ccr/auto_follow/A
{
  "remote_cluster": "<remote_cluster>",
  "leader_index_patterns": [
    "A*"
  ],
  "follow_index_pattern": "{{leader_index}}"
}

PUT /_ccr/auto_follow/0
{
  "remote_cluster": "<remote_cluster>",
  "leader_index_patterns": [
    "0*"
  ],
  "follow_index_pattern": "{{leader_index}}"
}

[elasticmachine]

Pinging @elastic/es-distributed (Team:Distributed)

[henningandersen]

There are really two parts here:

1. Not auto-following system indices. It seems like a bug that we allow this.
2. Exclusion pattern. We should add this as a separate field to the auto-follow request.

Both parts seems reasonable to add.

About the workaround, you could add just one auto follow pattern with the full list of single character patterns (still tedious for sure, but one pattern is easier to manage than many).

[fcofdez]

Closed via #72935 and #72815