[Fleet] Datastream rollovers for an integration do not rollover with the right namespace

[neptunian]

When upgrading a package, if mappings are incompatible, a rollover must take place on the data stream. The current query for data stream constant keywords to create the data stream name needs to be modified to account for unique namespaces, otherwise requests to the first namespace available will happen as many times as there are unique namespaces for the data stream.

We want to avoid parsing the indices to get the namespace in order to compose the data stream name, but we do not know what the namespace value is, hence the need for the query. What we need to solve for is the data stream name. After trying to update the mappings of each current write index of each data stream (this part is happening correctly), if one of those fails, prompting a rollover, compose the corresponding data stream by getting the namespace value and without parsing the index name for that value. We assumed there could be some risk in parsing the index name, but perhaps we should be open to it.

Perhaps we can think of a better way to query for the current indices of each data stream so that its corresponding constant keywords, like namespace, are returned with it.

Another option might be to keep a list of data streams needing rolled over and perform rollovers in a separate function. Dedup by the index pattern in the corresponding index template, and query to aggregate by namespace.

query

GET metrics-system.uptime-*/_search
{
  "size": 1,
  "_source": ["data_stream.namespace", "data_stream.type", "data_stream.dataset"],
  "query": {
    "bool": {
      "must": [
                {
          "exists": {
            "field": "data_stream.type"
          }
        },
        {
          "exists": {
            "field": "data_stream.namespace"
          }
        },
        {
          "exists": {
            "field": "data_stream.dataset"
          }
        }
      ]
    }
  },
  "aggs": {
    "index": {
      "terms": {
        "field": "data_stream.namespace"
      }
    }
  }
}

res

{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 7,
    "successful" : 7,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 2753,
      "relation" : "eq"
    },
    "max_score" : 3.0,
    "hits" : [
      {
        "_index" : ".ds-metrics-system.uptime-default-000001",
        "_type" : "_doc",
        "_id" : "6HjhQXcBVIIRiNxPUsqo",
        "_score" : 3.0,
        "_source" : {
          "data_stream" : {
            "namespace" : "default",
            "type" : "metrics",
            "dataset" : "system.uptime"
          }
        }
      }
    ]
  },
  "aggregations" : {
    "index" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [
        {
          "key" : "default",
          "doc_count" : 1589
        },
        {
          "key" : "namespace2",
          "doc_count" : 1164
        }
      ]
    }
  }
}

There are integration tests for rollovers, but not one that handles the use case of multiple namespaces

[elasticmachine]

Pinging @elastic/ingest-management (Team:Ingest Management)

[ph]

@neptunian @skh @ruflin This is indeed a good find. Thanks for the write up.