If an ingest node changes the target index, the original index is still created.

[clement-tourriere]

Elasticsearch version elasticsearch >= 6.5

Plugins installed: None

JVM version (11.0.1)

Date name processor creates prefixed index additionally to the date timed index.
This comportement has been introduced by this PR that creates the index before executing the pipeline: #32786

Steps to reproduce:

From a clean elasticsearch instance (from 6.5), just follow the example from date name processor documentation:

https://www.elastic.co/guide/en/elasticsearch/reference/current/date-index-name-processor.html

# Create the processor
curl -X PUT "localhost:9200/_ingest/pipeline/monthlyindex" -H 'Content-Type: application/json' -d'
{
  "description": "monthly date-time index naming",
  "processors" : [
    {
      "date_index_name" : {
        "field" : "date1",
        "index_name_prefix" : "myindex-",
        "date_rounding" : "M"
      }
    }
  ]
}
'

# Index a document

curl -X PUT "localhost:9200/myindex/_doc/1?pipeline=monthlyindex" -H 'Content-Type: application/json' -d'
{
  "date1" : "2016-04-25T12:02:01.789Z"
}
'


# my_index and myindex-2016-04-01 are created
curl -XGET "http://localhost:9200/_cat/indices?v"

[elasticmachine]

Pinging @elastic/es-core-features

[jakelandis]

Discussed this issue in person today.

The issues isn't specific to the Date Index Name processor, rather any processor that can change the location of _index. Below is an alternative reproduction:

DELETE foo,bar
GET _cat/indices?v

PUT _ingest/pipeline/test1
{
  "processors": [
    {
      "set": {
        "field": "_index",
        "value": "foo"
      }
    }
  ]
}

PUT bar/_doc/1?pipeline=test1
{
  "a" : "b"
}

GET foo/_doc/1
GET bar/_doc/1
GET _cat/indices?v

^ Note that both foo and bar indices are created.

#32786 is correctly identified as the change which introduces this issue. That change was made to ensure that a default pipeline, when defined via an index template would correctly find the default pipeline for the very first document as described in #32758.

The fix for this needs to ensure it does not re-introduce #32758. A likely fix for this will be:
if there is not an explicit pipeline requested AND the (request) target index does not exist
then change the order back to the original pipeline execution before index creation,
and then explicitly find and read the index template that will match the index request and pull the default pipeline (if it exists) from the index template.