UnsupportedOperationException in FetchDocValuesPhase with collapse

[frensjan]

Elasticsearch Version

Version: 8.14.3, Build: docker/d55f984299e0e88dee72ebd8255f7ff130859ad0/2024-07-07T22:04:49.882652950Z, JVM: 22.0.1

Installed Plugins

No response

Java Version

bundled

OS Version

Linux hostname 6.6.22-linuxkit #1 SMP Fri Mar 29 12:21:27 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux

Problem Description

In 8.14.2 a fix for a related issue was released, but I think a similar issue still remains. The script below runs fine against ES 7.x, but fails with ES 8.14.3 with an UnsupportedOperationException. The client is returned an error like this:

{"took":40,"timed_out":false,"_shards":{"total":10,"successful":3,"skipped":0,"failed":7,"failures":[
  {"shard":0,"index":"collapse-test","node":"l3Ukg0IrTxKyI_Fvj9fwZA","reason":{
    "type":"unsupported_operation_exception","reason":null
  }}
]},"hits":{"total":{"value":100,"relation":"eq"},"max_score":null,"hits":[]}}

Steps to Reproduce

Run ES, e.g. with a docker-compose.yml:

services:

  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.14.3
    environment:
      - xpack.security.enabled=false
      - discovery.type=single-node
      - network.bind_host=0.0.0.0
      - network.publish_host=127.0.0.1
    ports:
      - 127.0.0.1:9200:9200

Setup an index, add some docs and run a collapse query:

#!/usr/bin/env bash

set -e


echo "Deleting index (if exists)"
curl -X DELETE http://localhost:9200/collapse-test?ignore_unavailable=true
echo ; echo


echo "Creating index"
curl -X PUT http://localhost:9200/collapse-test -H 'Content-Type: application/json' -d '
  {
    "mappings": {
      "dynamic": "strict",
      "_source": {
        "excludes": [ "*" ]
      },
      "properties": {
        "url":  { "type": "keyword", "store": true },
        "ts":  { "type": "date", "store": true }
      }
    },
    "settings": {
      "number_of_shards": 10
    }
  }
'
echo ; echo


echo "Indexing data"
for i in {1..100} ; do
  curl -X POST http://localhost:9200/collapse-test/_doc -H 'Content-Type: application/json' -d "{
    \"url\": \"http://example.org/$i\",
    \"ts\": $i
  }"
  echo
done
echo


echo "Refreshing index"
curl -X POST http://localhost:9200/collapse-test/_refresh
echo ; echo


echo "Querying with collapse"
curl http://localhost:9200/collapse-test/_search -H 'Content-Type: application/json' -d '
  {
    "from": 0,
    "size": 16,
    "stored_fields": [
      "*"
    ],
    "sort": [
      {
        "ts": {
          "order": "desc"
        }
      }
    ],
    "collapse": {
      "field": "url"
    }
  }
  '
echo ; echo

Logs (if relevant)

shard [[l3Ukg0IrTxKyI_Fvj9fwZA][collapse-test][1]], reason [org.elasticsearch.transport.RemoteTransportException: [hostname][127.0.0.1:9300][indices:data/read/search[phase/fetch/id]]
Caused by: org.elasticsearch.search.fetch.FetchPhaseExecutionException: Fetch Failed [Error running fetch phase for doc [8]]
	at org.elasticsearch.server@8.14.3/org.elasticsearch.search.fetch.FetchPhaseDocsIterator.iterate(FetchPhaseDocsIterator.java:79)
	at org.elasticsearch.server@8.14.3/org.elasticsearch.search.fetch.FetchPhase.buildSearchHits(FetchPhase.java:187)
	at org.elasticsearch.server@8.14.3/org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:78)
	at org.elasticsearch.server@8.14.3/org.elasticsearch.search.SearchService.lambda$executeFetchPhase$9(SearchService.java:874)
	at org.elasticsearch.server@8.14.3/org.elasticsearch.action.ActionRunnable$3.accept(ActionRunnable.java:78)
...
Caused by: java.lang.UnsupportedOperationException
	at java.base/java.util.ImmutableCollections.uoe(ImmutableCollections.java:142)
	at java.base/java.util.ImmutableCollections$AbstractImmutableCollection.addAll(ImmutableCollections.java:148)
	at org.elasticsearch.server@8.14.3/org.elasticsearch.search.fetch.subphase.FetchDocValuesPhase$1.process(FetchDocValuesPhase.java:81)
	at org.elasticsearch.server@8.14.3/org.elasticsearch.search.fetch.FetchPhase$1.nextDoc(FetchPhase.java:175)
	at org.elasticsearch.server@8.14.3/org.elasticsearch.search.fetch.FetchPhaseDocsIterator.iterate(FetchPhaseDocsIterator.java:71)
	... 13 more

[elasticsearchmachine]

Pinging @elastic/es-search-relevance (Team:Search Relevance)

[benwtrent]

Indeed, the bug apparently is not fully fixed. I can replicate. Trying to figure out what I missed before.

[frensjan]

Thanks @benwtrent, good to see that a PR is already available. 👍