Filtering in non indexed field, as well as non existing field should return an exception

[gmoskovicz]

I have opened this pull request because to discuss if we can return an exception when filtering/querying a field that is not indexed, or that does not exist. As an example of this:

Querying a non-indexed field

PUT trial
{
  "mappings": {
    "document_type1": {
      "properties": {
        "prop1": {
          "index": "no",
          "type": "string"
        }
      }
    }
  }
}

POST trial/document_type1
{
  "prop1": "sure"
}

POST trial/_search
{
  "query": {
    "match": {
      "prop1": "sure"
    }
  }
}

This returns:

{
   "took": 13,
   "timed_out": false,
   "_shards": {
      "total": 5,
      "successful": 5,
      "failed": 0
   },
   "hits": {
      "total": 0,
      "max_score": null,
      "hits": []
   }
}

Querying a non-existing field

POST trial/_search
{
  "query": {
    "match": {
      "prop2": "some text"
    }
  }
}

This returns:

{
   "took": 13,
   "timed_out": false,
   "_shards": {
      "total": 5,
      "successful": 5,
      "failed": 0
   },
   "hits": {
      "total": 0,
      "max_score": null,
      "hits": []
   }
}

Both cases could return an exception, indicating that the field is not indexed, or does not exist. I've marked this issue as a discussion, since there are some cases such as querying multiple indices that could be a bit tricky to add consistency to the result of mixed results of existing fields on certain indices plus non-indexed/existing fields on other indices.

[tonyd]

Returning no results without any indication that the query could not be answered seems semantically incorrect - given a data store and a well-formed query, "no results" should mean that there are no matches to the query in that store; the alternative seems like an invitation to latent defects. I understand your point about multiple indices, but that case seems even worse - if one index is unable to respond to the query, you might get results, just not all of them. Perhaps there could be an option to allow it if, for example, you are migrating to a new index with a different mapping.

[clintongormley]

Duplicate of #12016

[helenzBV]

@tonyd Sorry to bother you but I just have a question regarding searching non-indexed field. I have informed by a colleague that it is possible to search non-indexed field, and he stated that he have seen this was being done by an elastic search expert. I'm curious if this is true. (Interestingly, this question is also a very controversial question all over internet). If this happens to be true, would you mind educate me on how to do it?

[tonyd]

@helenzBV Hi, no, I don't know of any way to do that - which doesn't mean there isn't one, but see #17748, which adds an option to give you an error if you search an unindexed field.