Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unable to cast IP DataType to String in Painless Script #22584

Closed
josephDunne opened this issue Jan 12, 2017 · 12 comments
Closed

Unable to cast IP DataType to String in Painless Script #22584

josephDunne opened this issue Jan 12, 2017 · 12 comments
Assignees
@nik9000
Labels
>bug :Core/Infra/Scripting Scripting abstractions, Painless, and Mustache

Comments

@josephDunne
Copy link

Elasticsearch version: 5.0

Plugins installed: default set

JVM version: openjdk version "1.8.0_101"

OS version: Centos 6

Description of the problem including expected versus actual behavior:

I have trying to append a field value to a string but I am getting an ArrayIndexOutOfBoundsException.
It seems that there is a problem with the .getValue() function for IP data type fields.

If anyone has a workaround I would much appreciate it.

This code does not work:

def ip = doc['IP_SRC'].getValue();

this code does not work:

def acc = 'my_ip:';
acc +=  doc['IP_SRC'];

Steps to reproduce:

  1. Create an field with mapping type ip
  2. Add documents some with doc values containing ip field data
  3. Try use that field in a string in a Painless script (I am using my script as a Kibana scripted field)

Provide logs (if relevant):

This log doesn't seem to show the cause of the Out of Bounds exception but I believe its somewhere in the to UTF8 routines of ByteRef.

[2017-01-12T12:37:49,043][DEBUG][o.e.a.s.TransportSearchAction] [Z8qG7Y5] [nap-gi-2017.01.12][0], node[Z8qG7Y5IScyM2r06BN0Dbg], [P], s[STARTED], a[id=TpRuYNq6TE-cY47uz3VlsA]: Failed to execute [SearchRequest{searchType=QUERY_AND_FETCH, indices=[nap-gi-2017.01.12], indicesOptions=IndicesOptions[id=39, ignore_unavailable=true, allow_no_indices=true, expand_wildcards_open=true, expand_wildcards_closed=false, allow_alisases_to_multiple_indices=true, forbid_closed_indices=true], types=[], routing='null', preference='1484222965683', requestCache=null, scroll=null, source={
  "size" : 500,
  "query" : {
    "bool" : {
      "must" : [
        {
          "query_string" : {
            "query" : "*",
            "fields" : [ ],
            "use_dis_max" : true,
            "tie_breaker" : 0.0,
            "default_operator" : "or",
            "auto_generate_phrase_queries" : false,
            "max_determined_states" : 10000,
            "lowercase_expanded_terms" : true,
            "enable_position_increment" : true,
            "fuzziness" : "AUTO",
            "fuzzy_prefix_length" : 0,
            "fuzzy_max_expansions" : 50,
            "phrase_slop" : 0,
            "analyze_wildcard" : true,
            "locale" : "und",
            "escape" : false,
            "boost" : 1.0
          }
        },
        {
          "range" : {
            "TS" : {
              "from" : 1484222823984,
              "to" : 1484224623984,
              "include_lower" : true,
              "include_upper" : true,
              "format" : "epoch_millis",
              "boost" : 1.0
            }
          }
        }
      ],
      "disable_coord" : false,
      "adjust_pure_negative" : true,
      "boost" : 1.0
    }
  },
  "_source" : {
    "includes" : [ ],
    "excludes" : [ ]
  },
  "stored_fields" : "*",
  "docvalue_fields" : [
    "TS"
  ],
  "script_fields" : {
    "search_token" : {
      "script" : {
        "inline" : "def json = '{';if (doc['IP_SRC'].size() > 0){\n  def asBytes = doc['IP_SRC'].getValue();\n  json += \"\\\"s\\\": \" + \",\";\n}\njson += '}';\nreturn json",
        "lang" : "painless"
      },
      "ignore_failure" : false
    }
  },
  "sort" : [
    {
      "TS" : {
        "order" : "desc",
        "unmapped_type" : "boolean"
      }
    }
  ],
  "aggregations" : {
    "2" : {
      "date_histogram" : {
        "field" : "TS",
        "time_zone" : "Europe/London",
        "interval" : "30s",
        "offset" : 0,
        "order" : {
          "_key" : "asc"
        },
        "keyed" : false,
        "min_doc_count" : 1
      }
    }
  },
  "highlight" : {
    "pre_tags" : [
      "@kibana-highlighted-field@"
    ],
    "post_tags" : [
      "@/kibana-highlighted-field@"
    ],
    "fragment_size" : 2147483647,
    "require_field_match" : false,
    "fields" : {
      "*" : { }
    }
  },
  "ext" : { }
}}]
org.elasticsearch.transport.RemoteTransportException: [Z8qG7Y5][127.0.0.1:9300][indices:data/read/search[phase/query+fetch]]
Caused by: org.elasticsearch.script.ScriptException: runtime error
        at org.elasticsearch.painless.ScriptImpl.convertToScriptException(ScriptImpl.java:177) ~[?:?]
        at org.elasticsearch.painless.ScriptImpl.run(ScriptImpl.java:124) ~[?:?]
        at org.elasticsearch.search.fetch.subphase.ScriptFieldsFetchSubPhase.hitExecute(ScriptFieldsFetchSubPhase.java:52) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:161) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:358) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.action.search.SearchTransportService.lambda$registerRequestHandler$9(SearchTransportService.java:291) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.transport.TransportRequestHandler.messageReceived(TransportRequestHandler.java:33) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.transport.TransportService$6.doRun(TransportService.java:548) [elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:504) [elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-5.0.0.jar:5.0.0]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_101]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_101]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_101]
Caused by: java.lang.ArrayIndexOutOfBoundsException
[2017-01-12T12:37:49,045][DEBUG][o.e.a.s.TransportSearchAction] [Z8qG7Y5] All shards failed for phase: [query_fetch]
org.elasticsearch.transport.RemoteTransportException: [Z8qG7Y5][127.0.0.1:9300][indices:data/read/search[phase/query+fetch]]
Caused by: org.elasticsearch.script.ScriptException: runtime error
        at org.elasticsearch.painless.ScriptImpl.convertToScriptException(ScriptImpl.java:177) ~[?:?]
        at org.elasticsearch.painless.ScriptImpl.run(ScriptImpl.java:124) ~[?:?]
        at org.elasticsearch.search.fetch.subphase.ScriptFieldsFetchSubPhase.hitExecute(ScriptFieldsFetchSubPhase.java:52) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:161) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:358) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.action.search.SearchTransportService.lambda$registerRequestHandler$9(SearchTransportService.java:291) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.transport.TransportRequestHandler.messageReceived(TransportRequestHandler.java:33) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.transport.TransportService$6.doRun(TransportService.java:548) [elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:504) [elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-5.0.0.jar:5.0.0]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_101]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_101]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_101]
Caused by: java.lang.ArrayIndexOutOfBoundsException
@nik9000
Copy link
Member

nik9000 commented Jan 12, 2017

That stacktrace looks cut off. If that is how it is logged then we have another problem in addition to whatever you've found with ips. Also look out for #22393.

@josephDunne
Copy link
Author

That is indeed how it is logged :) if I mess around with the script I abitni can sometimes get a longer stack trace which shows the source of the out of bounds in the UTF8 code

@josephDunne
Copy link
Author

Funny you should mention #22393 as I had run into the string escape issue and I have seen that ticket. It does not seem to solve this issue

@nik9000
Copy link
Member

nik9000 commented Jan 12, 2017

That is indeed how it is logged :)

Great! We play some tricks with those stacktraces in an effort to make them a bit more useable but it looks like we've gone a bit too far.

Funny you should mention #22393 as I had run into the string escape issue and I have seen that ticket. It does not seem to solve this issue

Yeah. I just meant that you would have to work around that issue in addition to any workaround we find for this one.

I'll start in on this as soon as I can get finished replying to other github issues.

@nik9000 nik9000 self-assigned this Jan 12, 2017
@josephDunne
Copy link
Author

thank you @nik9000! My hunch is that there needs to be a specific subclass in ScriptDocValues for IP fields. Here is a longer trace:

Caused by: org.elasticsearch.script.ScriptException: runtime error
        at org.elasticsearch.painless.ScriptImpl.convertToScriptException(ScriptImpl.java:177) ~[?:?]
        at org.elasticsearch.painless.ScriptImpl.run(ScriptImpl.java:124) ~[?:?]
        at org.elasticsearch.search.fetch.subphase.ScriptFieldsFetchSubPhase.hitExecute(ScriptFieldsFetchSubPhase.java:52) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:161) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:358) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.action.search.SearchTransportService.lambda$registerRequestHandler$9(SearchTransportService.java:291) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.transport.TransportRequestHandler.messageReceived(TransportRequestHandler.java:33) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.transport.TransportService$6.doRun(TransportService.java:548) [elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:504) [elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-5.0.0.jar:5.0.0]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_101]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_101]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_101]
Caused by: java.lang.ArrayIndexOutOfBoundsException: 16
        at org.apache.lucene.util.UnicodeUtil.UTF8toUTF16(UnicodeUtil.java:598) ~[lucene-core-6.2.0.jar:6.2.0 764d0f19151dbff6f5fcd9fc4b2682cf934590c5 - mike - 2016-08-20 05:39:36]
        at org.apache.lucene.util.BytesRef.utf8ToString(BytesRef.java:152) ~[lucene-core-6.2.0.jar:6.2.0 764d0f19151dbff6f5fcd9fc4b2682cf934590c5 - mike - 2016-08-20 05:39:36]
        at org.elasticsearch.index.fielddata.ScriptDocValues$Strings.getValue(ScriptDocValues.java:84) ~[elasticsearch-5.0.0.jar:5.0.0]
        at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:627) ~[?:1.8.0_101]
        at org.elasticsearch.painless.DefBootstrap$PIC.fallback(DefBootstrap.java:204) ~[?:?]
        at org.elasticsearch.painless.Executable$Script.execute(def json = '{'; ... @ <inline script>:332) ~[?:?]
        at org.elasticsearch.painless.ScriptImpl.run(ScriptImpl.java:121) ~[?:?]
        at org.elasticsearch.search.fetch.subphase.ScriptFieldsFetchSubPhase.hitExecute(ScriptFieldsFetchSubPhase.java:52) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:161) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:358) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.action.search.SearchTransportService.lambda$registerRequestHandler$9(SearchTransportService.java:291) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.transport.TransportRequestHandler.messageReceived(TransportRequestHandler.java:33) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.transport.TransportService$6.doRun(TransportService.java:548) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:504) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-5.0.0.jar:5.0.0]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_101]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_101]
        at java.lang.Thread.run(Thread.java:745) ~[?:1.8.0_101]
[2017-01-12T12:14:30,913][DEBUG][o.e.a.s.TransportSearchAction] [Z8qG7Y5] All shards failed for phase: [query_fetch]
org.elasticsearch.transport.RemoteTransportException: [Z8qG7Y5][127.0.0.1:9300][indices:data/read/search[phase/query+fetch]]
Caused by: org.elasticsearch.script.ScriptException: runtime error
        at org.elasticsearch.painless.ScriptImpl.convertToScriptException(ScriptImpl.java:177) ~[?:?]
        at org.elasticsearch.painless.ScriptImpl.run(ScriptImpl.java:124) ~[?:?]
        at org.elasticsearch.search.fetch.subphase.ScriptFieldsFetchSubPhase.hitExecute(ScriptFieldsFetchSubPhase.java:52) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:161) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:358) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.action.search.SearchTransportService.lambda$registerRequestHandler$9(SearchTransportService.java:291) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.transport.TransportRequestHandler.messageReceived(TransportRequestHandler.java:33) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.transport.TransportService$6.doRun(TransportService.java:548) [elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:504) [elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-5.0.0.jar:5.0.0]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_101]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_101]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_101]
Caused by: java.lang.ArrayIndexOutOfBoundsException: 16
        at org.apache.lucene.util.UnicodeUtil.UTF8toUTF16(UnicodeUtil.java:598) ~[lucene-core-6.2.0.jar:6.2.0 764d0f19151dbff6f5fcd9fc4b2682cf934590c5 - mike - 2016-08-20 05:39:36]
        at org.apache.lucene.util.BytesRef.utf8ToString(BytesRef.java:152) ~[lucene-core-6.2.0.jar:6.2.0 764d0f19151dbff6f5fcd9fc4b2682cf934590c5 - mike - 2016-08-20 05:39:36]
        at org.elasticsearch.index.fielddata.ScriptDocValues$Strings.getValue(ScriptDocValues.java:84) ~[elasticsearch-5.0.0.jar:5.0.0]
        at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:627) ~[?:1.8.0_101]
        at org.elasticsearch.painless.DefBootstrap$PIC.fallback(DefBootstrap.java:204) ~[?:?]
        at org.elasticsearch.painless.Executable$Script.execute(def json = '{'; ... @ <inline script>:332) ~[?:?]
        at org.elasticsearch.painless.ScriptImpl.run(ScriptImpl.java:121) ~[?:?]
        at org.elasticsearch.search.fetch.subphase.ScriptFieldsFetchSubPhase.hitExecute(ScriptFieldsFetchSubPhase.java:52) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:161) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:358) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.action.search.SearchTransportService.lambda$registerRequestHandler$9(SearchTransportService.java:291) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.transport.TransportRequestHandler.messageReceived(TransportRequestHandler.java:33) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.transport.TransportService$6.doRun(TransportService.java:548) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:504) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-5.0.0.jar:5.0.0]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_101]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_101]
        at java.lang.Thread.run(Thread.java:745) ~[?:1.8.0_101]

I got that with this script and don't understand really what makes it different :-) :

 String ip = doc['IP_SRC'].getValue();

Also I am not sure if it helps or if this is a different issue but if I do the following script I get a NoSuchElement exception (I thought maybe its because the field is null but even if i wrap it in a if statement i get the same issue):

def mystring = 'myip:';
mystring += doc['IP_SRC'];

org.elasticsearch.transport.RemoteTransportException: [Z8qG7Y5][127.0.0.1:9300][indices:data/read/search[phase/query+fetch]]
Caused by: org.elasticsearch.script.ScriptException: runtime error
        at org.elasticsearch.painless.ScriptImpl.convertToScriptException(ScriptImpl.java:177) ~[?:?]
        at org.elasticsearch.painless.ScriptImpl.run(ScriptImpl.java:124) ~[?:?]
        at org.elasticsearch.search.fetch.subphase.ScriptFieldsFetchSubPhase.hitExecute(ScriptFieldsFetchSubPhase.java:52) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:161) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:358) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.action.search.SearchTransportService.lambda$registerRequestHandler$9(SearchTransportService.java:291) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.transport.TransportRequestHandler.messageReceived(TransportRequestHandler.java:33) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.transport.TransportService$6.doRun(TransportService.java:548) [elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:504) [elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-5.0.0.jar:5.0.0]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_101]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_101]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_101]
Caused by: java.util.NoSuchElementException
        at java.util.AbstractList$Itr.next(AbstractList.java:364) ~[?:1.8.0_101]
        at java.util.AbstractCollection.toString(AbstractCollection.java:461) ~[?:1.8.0_101]
        at java.lang.String.valueOf(String.java:2994) ~[?:1.8.0_101]
        at java.lang.StringBuilder.append(StringBuilder.java:131) ~[?:1.8.0_101]
        at org.elasticsearch.painless.Executable$Script.execute(def json = '{'; ... @ <inline script>:325) ~[?:?]
        at org.elasticsearch.painless.ScriptImpl.run(ScriptImpl.java:121) ~[?:?]
        at org.elasticsearch.search.fetch.subphase.ScriptFieldsFetchSubPhase.hitExecute(ScriptFieldsFetchSubPhase.java:52) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:161) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:358) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.action.search.SearchTransportService.lambda$registerRequestHandler$9(SearchTransportService.java:291) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.transport.TransportRequestHandler.messageReceived(TransportRequestHandler.java:33) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.transport.TransportService$6.doRun(TransportService.java:548) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:504) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-5.0.0.jar:5.0.0]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_101]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_101]
        at java.lang.Thread.run(Thread.java:745) ~[?:1.8.0_101]
[2017-01-12T15:07:50,668][DEBUG][o.e.a.s.TransportSearchAction] [Z8qG7Y5] All shards failed for phase: [query_fetch]
org.elasticsearch.transport.RemoteTransportException: [Z8qG7Y5][127.0.0.1:9300][indices:data/read/search[phase/query+fetch]]
Caused by: org.elasticsearch.script.ScriptException: runtime error
        at org.elasticsearch.painless.ScriptImpl.convertToScriptException(ScriptImpl.java:177) ~[?:?]
        at org.elasticsearch.painless.ScriptImpl.run(ScriptImpl.java:124) ~[?:?]
        at org.elasticsearch.search.fetch.subphase.ScriptFieldsFetchSubPhase.hitExecute(ScriptFieldsFetchSubPhase.java:52) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:161) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:358) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.action.search.SearchTransportService.lambda$registerRequestHandler$9(SearchTransportService.java:291) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.transport.TransportRequestHandler.messageReceived(TransportRequestHandler.java:33) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.transport.TransportService$6.doRun(TransportService.java:548) [elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:504) [elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-5.0.0.jar:5.0.0]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_101]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_101]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_101]
Caused by: java.util.NoSuchElementException
        at java.util.AbstractList$Itr.next(AbstractList.java:364) ~[?:1.8.0_101]
        at java.util.AbstractCollection.toString(AbstractCollection.java:461) ~[?:1.8.0_101]
        at java.lang.String.valueOf(String.java:2994) ~[?:1.8.0_101]
        at java.lang.StringBuilder.append(StringBuilder.java:131) ~[?:1.8.0_101]
        at org.elasticsearch.painless.Executable$Script.execute(def json = '{'; ... @ <inline script>:325) ~[?:?]
        at org.elasticsearch.painless.ScriptImpl.run(ScriptImpl.java:121) ~[?:?]
        at org.elasticsearch.search.fetch.subphase.ScriptFieldsFetchSubPhase.hitExecute(ScriptFieldsFetchSubPhase.java:52) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.search.fetch.FetchPhase.execute(FetchPhase.java:161) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.search.SearchService.executeFetchPhase(SearchService.java:358) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.action.search.SearchTransportService.lambda$registerRequestHandler$9(SearchTransportService.java:291) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.transport.TransportRequestHandler.messageReceived(TransportRequestHandler.java:33) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.transport.TransportService$6.doRun(TransportService.java:548) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:504) ~[elasticsearch-5.0.0.jar:5.0.0]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-5.0.0.jar:5.0.0]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_101]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_101]
        at java.lang.Thread.run(Thread.java:745) ~[?:1.8.0_101]

@nik9000
Copy link
Member

nik9000 commented Jan 12, 2017

My hunch is that there needs to be a specific subclass in ScriptDocValues for IP fields.

Looks like @jpountz already got it: cbd4340

but there still is an issue in master:

Unable to find dynamic method [getValue] with [0] arguments for class [org.elasticsearch.index.mapper.IpFieldMapper.IpFieldType.IpScriptDocValues]

It looks like we're missing a whitelist entry for painless. Digging....

@nik9000
Copy link
Member

nik9000 commented Jan 12, 2017

Ok - I've almost got it but I can't figure out why painless can't find a class. I'm going to take a break and come back in a bit.

@jdconrad
Copy link
Contributor

@nik9000 I'm not seeing this in the whitelist in master.

@nik9000
Copy link
Member

nik9000 commented Jan 12, 2017

@nik9000 I'm not seeing this in the whitelist in master.

Yeah, I'm whitelisting it now and making a test for them all. But I'm having trouble with class not found exceptions.

@jdconrad
Copy link
Contributor

Ahh, sorry I misunderstood.

nik9000 added a commit to nik9000/elasticsearch that referenced this issue Jan 12, 2017
@nik9000
Whitelist some ScriptDocValues in painless
499b9a3 
Without this whitelist painless can't use ip or binary doc values.

Closes elastic#22584
@nik9000 nik9000 mentioned this issue Jan 12, 2017
Merged
nik9000 added a commit that referenced this issue Jan 12, 2017
@nik9000
Whitelist some ScriptDocValues in painless (#22600)
baed02b 
Without this whitelist painless can't use ip or binary doc values.

Closes #22584
nik9000 added a commit that referenced this issue Jan 12, 2017
@nik9000
Whitelist some ScriptDocValues in painless (#22600)
8b052ac 
Without this whitelist painless can't use ip or binary doc values.

Closes #22584
nik9000 added a commit that referenced this issue Jan 12, 2017
@nik9000
Whitelist some ScriptDocValues in painless (#22600)
7e5ac70 
Without this whitelist painless can't use ip or binary doc values.

Closes #22584
@josephDunne
Copy link
Author

Thank you @nik9000

@nik9000
Copy link
Member

nik9000 commented Jan 12, 2017

Thank you @nik9000

You are quite welcome! You'll get it in 5.2.0 which isn't quite ready yet, but is coming soonish.

@clintongormley clintongormley added :Core/Infra/Scripting Scripting abstractions, Painless, and Mustache and removed :Plugin Lang Painless labels Feb 14, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@nik9000 nik9000

Labels
>bug :Core/Infra/Scripting Scripting abstractions, Painless, and Mustache
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@clintongormley @nik9000 @josephDunne @jdconrad