[RAC] Index naming: change the way index prefix works

[banderror]

Parent ticket: #101016

Summary

Currently, all alerts-as-data indices share a default prefix .alerts, so all the indices end up being

.alerts-security.alerts-{namespace}
.alerts-observability.logs.alerts-{namespace}
.alerts-observability.logs.events-{namespace}
etc...

Kibana admin is able to override this prefix via Kibana config to something like .kibana_mytenant_alerts or anyotherprefix:

xpack.ruleRegistry.index: .kibana_mytenant_alerts

// in this case, index names would be
.kibana_mytenant_alerts-security.alerts-{namespace}
.kibana_mytenant_alerts-observability.logs.alerts-{namespace}
.kibana_mytenant_alerts-observability.logs.events-{namespace}

We were planning to support legacy multi-tenancy via this setting, but after implementing RBAC for alerts, decided to give users less degree of freedom. It would be much easier for RBAC if all alerts-as-data indices started with .alerts- no matter what.

So, we should change the implementation of index prefix in this way:

xpack.ruleRegistry.index: whatever

// in this case, index names should be
.alerts-whatever-security.alerts-{namespace}
.alerts-whatever-observability.logs.alerts-{namespace}
.alerts-whatever-observability.logs.events-{namespace}

If xpack.ruleRegistry.index is not specified or blank, we should default to .alerts:

.alerts-security.alerts-{namespace}
.alerts-observability.logs.alerts-{namespace}
.alerts-observability.logs.events-{namespace}

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[banderror]

@dhurley14 @jasonrhodes let me know if I misunderstood the requirements for this one

[jasonrhodes]

This is right except we should not use - to separate .alerts with the config value, because we rely on things being in the right order in between the dashes.

[elasticmachine]

Pinging @elastic/logs-metrics-ui (Team:logs-metrics-ui)

[weltenwort]

This looks good to me (if we leave aside the .internal-alerts discussion). 👍 I just want to point out that we have hard-coded index prefixes in the RBAC code at the moment, which doesn't work with the configurable prefix:

kibana/packages/kbn-rule-data-utils/src/alerts_as_data_rbac.ts

Lines 28 to 35 in 402702d

	export const mapConsumerToIndexName: Record<ALERTS_CONSUMERS, string | string[]> = {
	apm: '.alerts-observability-apm',
	logs: '.alerts-observability.logs',
	infrastructure: '.alerts-observability.metrics',
	observability: '.alerts-observability',
	siem: ['.alerts-security.alerts', '.siem-signals'],
	synthetics: '.alerts-observability-synthetics',
	};

[jasonrhodes]

@dhurley14 / @yctercero / @peluja1012 -- what are your thoughts on how this works (doesn't work) alongside RBAC?

[yctercero]

@jasonrhodes this is the issue that was brought up a few weeks ago, but the discussion kind of fizzled out - #104958

Totally agree that hardcoding is not how we want to have it. We were trying to get things in for 7.14 and with ruleRegistry becoming a library, lost our source of state per se. I think @marshallmain had mentioned during a meeting that each solution could register the alerts index with the rule registry, but I'm forgetting a bit what else that entailed as there does need to be a central location for RBAC to be aware of all solution indices.

I recall there being a few things discussed. We could 1) add state back into the rule registry or 2) find if there's a way to register this with alerting, as RBAC already pulls from it and I think the consensus was that this will eventually end up in there?

[jasonrhodes]

@yctercero thanks! On the one hand, I'm absolutely on board with your option (1), adding some state back into the rule registry. As long as we don't block access to the Alerting Framework's types and APIs, any state that's specific to RAC can be stored in rule registry as far as I'm concerned.

That said, for this, I think we just need the RBAC code to be able to read from the config value so that the RBAC can be set up to respect this config value, for now. So we may not need to solve this whole state issue in order to make sure RBAC can work with the "legacy multi-tenancy" situation of letting a user specify a prefix for their indices.

[weltenwort]

That said, for this, I think we just need the RBAC code to be able to read from the config value so that the RBAC can be set up to respect this config value, for now. So we may not need to solve this whole state issue in order to make sure RBAC can work with the "legacy multi-tenancy" situation of letting a user specify a prefix for their indices.

That sounds plausible. The RuleDataPluginService instantiated by the rule_registry already knows the index prefix and could provide a method to derive a full alias name given a registrationContext, eventType and namespace, which can serve as the single source of truth.

[jasonrhodes]

This work is not possible without significant risk, so we have opted for a different solution: #108506