Handle index template with no template defined correctly

[martijnvg]

A composable index template with no template defined in the body is mistakingly always assumed to not be a time series template. Even if it refers to a component template that has the index.mode setting set to time_series and the component template defines mappings with dimension fields or routing paths.

Incorrectly this causes to create a data stream with a tsdb index without time series start and end time settings set. The fact that this can happen is another bug and is being tracked #98833.

What should happen in this case is that a tsdb data stream is created with a valid tsdb backing index (with correctly generated time series start and end time settings).

This bug was found as part investigating #98792 and is the cause of that bug.

[elasticsearchmachine]

Pinging @elastic/es-analytics-geo (Team:Analytics)

[martijnvg]

Reproduction:

PUT /_component_template/component_template_1
{
    "template": {
        "settings": {
            "mode": "time_series",
            "routing_path": [
                "location"
            ]
        },
        "mappings": {
            "properties": {
                "location": {
                    "type": "keyword",
                    "time_series_dimension": true
                }
            }
        }
    }
}

PUT _index_template/index_template_1
{
    "index_patterns": [
        "test*"
    ],
    "data_stream": {},
    "composed_of": [
        "component_template_1"
    ]
}

POST test1/_doc
{
    "@timestamp": "2023-08-25T05:00:00.000Z",
    "location": "abc"
}

Executing get data stream api: GET /_data_stream results into:

{
    "data_streams": [
        {
            "name": "test1",
            "timestamp_field": {
                "name": "@timestamp"
            },
            "indices": [
                {
                    "index_name": ".ds-test1-2023.08.25-000001",
                    "index_uuid": "zs7Y5WGERLKcr3jNqJLVtw"
                }
            ],
            "generation": 1,
            "status": "YELLOW",
            "template": "index_template_1",
            "lifecycle": {
                "enabled": true
            },
            "hidden": false,
            "system": false,
            "allow_custom_routing": false,
            "replicated": false
        }
    ]
}

The data stream isn't a tsdb data stream (the time_series is missing).
But when checking the get index api for one of the backing indices: GET .ds-test1-2023.08.25-000001 results into:

{
    ".ds-test1-2023.08.25-000001": {
        "aliases": {},
        "mappings": {
            "_data_stream_timestamp": {
                "enabled": true
            },
            "properties": {
                "@timestamp": {
                    "type": "date"
                },
                "location": {
                    "type": "keyword",
                    "time_series_dimension": true
                }
            }
        },
        "settings": {
            "index": {
                "mode": "time_series",
                "routing": {
                    "allocation": {
                        "include": {
                            "_tier_preference": "data_hot"
                        }
                    }
                },
                "hidden": "true",
                "number_of_shards": "1",
                "provided_name": ".ds-test1-2023.08.25-000001",
                "creation_date": "1692933138275",
                "number_of_replicas": "1",
                "routing_path": [
                    "location"
                ],
                "uuid": "zs7Y5WGERLKcr3jNqJLVtw",
                "version": {
                    "created": "8110099"
                }
            }
        },
        "data_stream": "test1"
    }
}

The index.mode setting is set, which does make it a tsdb index.
But the index.time_series.start_time and index.time_series.end_time index settings are missing.
The fact that such tsdb indices can be created is another bug tracked here: #98833

[martijnvg]

A workaround if deleting the invalid data stream is acceptable, would be to add an empty template object field to the index template with name index_template_1 :

PUT /_component_template/component_template_1
{
    "template": {
        "settings": {
            "mode": "time_series",
            "routing_path": [
                "location"
            ]
        },
        "mappings": {
            "properties": {
                "location": {
                    "type": "keyword",
                    "time_series_dimension": true
                }
            }
        }
    }
}

PUT _index_template/index_template_1
{
    "index_patterns": [
        "test*"
    ],
    "template": {},
    "data_stream": {},
    "composed_of": [
        "component_template_1"
    ]
}

POST test1/_doc
{
    "@timestamp": "2023-08-25T05:00:00.000Z",
    "location": "abc"
}

When executing get data stream ap nowi: GET /_data_stream results into:

{
    "data_streams": [
        {
            "name": "test1",
            "timestamp_field": {
                "name": "@timestamp"
            },
            "indices": [
                {
                    "index_name": ".ds-test1-2023.08.25-000001",
                    "index_uuid": "0EYNPGwNTCe8dyG8wR7SWg"
                }
            ],
            "generation": 1,
            "status": "YELLOW",
            "template": "index_template_1",
            "lifecycle": {
                "enabled": true
            },
            "hidden": false,
            "system": false,
            "allow_custom_routing": false,
            "replicated": false,
            "time_series": {
                "temporal_ranges": [
                    {
                        "start": "2023-08-25T01:15:47.000Z",
                        "end": "2023-08-25T05:20:58.000Z"
                    }
                ]
            }
        }
    ]
}

The time_series section is now present.