Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ensures watch definitions are valid json #30692

Closed
wants to merge 5 commits into from
Closed

Ensures watch definitions are valid json #30692

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
36d0c12
Ensures watch definitions are valid json
May 17, 2018
6da9e8b
address reviewer feedback
May 17, 2018
7252b00
address reviewer feedback
May 17, 2018
9a4db9f
address reviewer feedback
tomcallahan May 23, 2018
dcd9b86
Merge remote-tracking branch 'origin/master' into ensure-watch-valid-…
tomcallahan May 24, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions x-pack/plugin/watcher/src/main/java/org/elasticsearch/xpack/watcher/watch/WatchParser.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
*/
package org.elasticsearch.xpack.watcher.watch;

import com.fasterxml.jackson.core.JsonParseException;
import org.elasticsearch.ElasticsearchParseException;
import org.elasticsearch.common.Nullable;
import org.elasticsearch.common.bytes.BytesReference;
Expand Down Expand Up @@ -174,6 +175,13 @@ public Watch parse(String id, boolean includeStatus, WatcherXContentParser parse
throw new ElasticsearchParseException("could not parse watch [{}]. unexpected field [{}]", id, currentFieldName);
}
}
/* Make sure we are at the end of the available input data -- certain types of JSON errors will not manifest
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor nit: can you use a // comment like the other comments in the methods as well and add new lines before and after?

until we try to consume additional tokens.
*/
if (parser.nextToken() != null) {
throw new ElasticsearchParseException("could not parse watch [{}]. unexpected data beyond [line: {}, column: {}]",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor nit: maybe we can come up with a more explaining error message here, as unexpected data can mean anything - however I don't have a smarter idea as well to be honest. Is it unparseable/unknown or just too much data/unneeded additional data?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@spinscale Actually it is this code that enables the JsonParseException for the original test case. Ryan asked that the code for the test case be changed, which resulted in a different error path. However, the key factor is that we now attempt to grab the next token beyond what we believe to be the last token of the watch definition. A previous revision of this code made that fact clear, at the expense of some more complex code.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yea I think what @spinscale is saying is that he would like the error message changed. Maybe expected end of payload/data/json but received additional data

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hub-cap I was trying to respond to @spinscale's comment above in the LGTM -- I'll rework the error message

id, parser.getTokenLocation().lineNumber, parser.getTokenLocation().columnNumber);
}
if (trigger == null) {
throw new ElasticsearchParseException("could not parse watch [{}]. missing required field [{}]", id,
WatchField.TRIGGER.getPreferredName());
Expand Down
32 changes: 32 additions & 0 deletions x-pack/plugin/watcher/src/test/java/org/elasticsearch/xpack/watcher/watch/WatchTests.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,18 @@
import org.elasticsearch.action.support.WriteRequest;
import org.elasticsearch.client.Client;
import org.elasticsearch.common.ParseField;
import org.elasticsearch.common.bytes.BytesArray;
import org.elasticsearch.common.bytes.BytesReference;
import org.elasticsearch.common.logging.Loggers;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.unit.TimeValue;
import org.elasticsearch.common.xcontent.DeprecationHandler;
import org.elasticsearch.common.xcontent.NamedXContentRegistry;
import org.elasticsearch.common.xcontent.XContentBuilder;
import org.elasticsearch.common.xcontent.XContentParser;
import org.elasticsearch.common.xcontent.XContentType;
import org.elasticsearch.common.xcontent.json.JsonXContent;
import org.elasticsearch.common.xcontent.json.JsonXContentParser;
import org.elasticsearch.index.query.MatchAllQueryBuilder;
import org.elasticsearch.index.query.QueryBuilder;
import org.elasticsearch.index.query.ScriptQueryBuilder;
Expand Down Expand Up @@ -121,6 +125,7 @@
import org.junit.Before;

import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.time.Clock;
import java.time.Instant;
import java.time.ZoneOffset;
Expand All @@ -143,6 +148,8 @@
import static org.elasticsearch.xpack.watcher.input.InputBuilders.searchInput;
import static org.elasticsearch.xpack.watcher.test.WatcherTestUtils.templateRequest;
import static org.elasticsearch.xpack.watcher.trigger.TriggerBuilders.schedule;

import static org.hamcrest.Matchers.containsString;
import static org.hamcrest.Matchers.equalTo;
import static org.hamcrest.Matchers.hasSize;
import static org.hamcrest.Matchers.instanceOf;
Expand Down Expand Up @@ -296,6 +303,31 @@ public void testParserBadActions() throws Exception {
}
}

public void testParserConsumesEntireDefinition() throws Exception {
WatchParser wp = createWatchparser();
XContentBuilder jsonBuilder = jsonBuilder()
.startObject()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason you did not opt for the syntax (yea i know, its a builder, lets chain it!!!) used in other spots in this class? Imho its harder to read the nested thing than blocks of code with spaces that define start/endings. This is also how we do it in most of our toXContent functions as well.

b = jsonBuilder();
b.startObject();

b.startObject("trigger");
b.startObject("schedule");
b.field("interval", "10s");
b.endObject();
b.endObject();

...

b.endObject();

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We prefer using braces like this:

diff --git a/x-pack/plugin/watcher/src/test/java/org/elasticsearch/xpack/watcher/watch/WatchTests.java b/x-pack/plugin/watcher/src/test/java/org/elasticsearch/xpack/watcher/watch/WatchTests.java
index 1ff59034831..dcb48fbd3c1 100644
--- a/x-pack/plugin/watcher/src/test/java/org/elasticsearch/xpack/watcher/watch/WatchTests.java
+++ b/x-pack/plugin/watcher/src/test/java/org/elasticsearch/xpack/watcher/watch/WatchTests.java
@@ -305,27 +305,44 @@ public class WatchTests extends ESTestCase {
 
     public void testParserConsumesEntireDefinition() throws Exception {
         WatchParser wp = createWatchparser();
-        XContentBuilder jsonBuilder = jsonBuilder()
-            .startObject()
-              .startObject("trigger")
-                .startObject("schedule")
-                  .field("interval", "10s")
-                .endObject()
-              .endObject()
-              .startObject("input")
-                .startObject("simple")
-                .endObject()
-              .endObject()
-              .startObject("condition")
-                .startObject("script")
-                  .field("source", "return false")
-                .endObject()
-              .endObject()
-            .endObject();
-            jsonBuilder.generator().writeBinary(",".getBytes(StandardCharsets.UTF_8));
-            ElasticsearchParseException e = expectThrows(ElasticsearchParseException.class, () ->
-                wp.parseWithSecrets("failure", false, BytesReference.bytes(jsonBuilder), new DateTime(), XContentType.JSON, true));
+        try (XContentBuilder builder = jsonBuilder()) {
+            builder.startObject();
+            {
+                builder.startObject("trigger");
+                {
+                    builder.startObject("schedule");
+                    {
+                        builder.field("interval", "10s");
+                    }
+                    builder.endObject();
+                }
+                builder.endObject();
+                builder.startObject("input");
+                {
+                    builder.startObject("simple");
+                    {
+                    }
+                    builder.endObject();
+                }
+                builder.endObject();
+                builder.startObject("condition");
+                {
+                    builder.startObject("script");
+                    {
+                        builder.field("source", "return false");
+                    }
+                    builder.endObject();
+                }
+                builder.endObject();
+            }
+            builder.endObject();
+            builder.generator().writeBinary(",".getBytes(StandardCharsets.UTF_8));
+            ElasticsearchParseException e = expectThrows(
+                    ElasticsearchParseException.class,
+                    () -> wp.parseWithSecrets("failure", false, BytesReference.bytes(builder), new DateTime(), XContentType.JSON, true));
             assertThat(e.getMessage(), containsString("unexpected data beyond"));
+        }
+
     }
 
     public void testParserDefaults() throws Exception {

Also note that the builder should be wrapped in a try-with-resources; it's good hygiene.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, ive not see this before but i like it. I almost removed a builder.endObject() on one of my first XContent forays, and I dont think i would have even considered it if this syntax was used.

Also, WRT the try-with-resources and hygiene, this is specifically for tests? or should we be doing this in the XContent methods as well? I assumed the caller was responsible for it in our request/response objects.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we own the lifecycle of the builder so we are responsible for closing it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hub-cap The way the class does it is mixed, when i started this PR i based style on testParserBadActions(), as that seemed to be the closest example to what I wanted to do. I am happy to conform to whatever is preferred.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's the old style before we introduced using braces for this.

.startObject("trigger")
.startObject("schedule")
.field("interval", "10s")
.endObject()
.endObject()
.startObject("input")
.startObject("simple")
.endObject()
.endObject()
.startObject("condition")
.startObject("script")
.field("source", "return false")
.endObject()
.endObject()
.endObject();
jsonBuilder.generator().writeBinary(",".getBytes(StandardCharsets.UTF_8));
ElasticsearchParseException e = expectThrows(ElasticsearchParseException.class, () ->
wp.parseWithSecrets("failure", false, BytesReference.bytes(jsonBuilder), new DateTime(), XContentType.JSON, true));
assertThat(e.getMessage(), containsString("unexpected data beyond"));
}

public void testParserDefaults() throws Exception {
Schedule schedule = randomSchedule();
ScheduleRegistry scheduleRegistry = registry(schedule);
Expand Down