Make Geo Context Mapping Parsing More Strict #32821
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Currently, if geo context is represented by something other than geo_point or an object with lat and lon fields, the parsing of it as a geo context can result in ignoring the context altogether, returning confusing errors such as number_format_exception or trying to parse the number specifying as long-encoded hash code. It would also fail if the geo_point was stored. This commit makes the mapping parsing more strict and will fail during mapping update or index creation if the geo context doesn't point to a geo_point field. Supersedes elastic#32683 Closes elastic#32202
imotov
added
>bug
>breaking
:Analytics/Geo
|
Pinging @elastic/es-search-aggs |
|
@jimczi yes, I was thinking about backporting it to 6.x, but I think I should open a separate PR for it since there are some non-trivial differences comparing to the master version. |
jtibshirani
reviewed
Aug 13, 2018
| } else { | ||
| // todo return this to .stringValue() once LatLonPoint implements it | ||
| geohashes.add(spare.geohash()); | ||
| } if (field instanceof LatLonPoint || field instanceof LatLonDocValuesField) { |
There was a problem hiding this comment.
Should there be an else here?
| @@ -421,6 +422,8 @@ static void validateTypeName(String type) { | |||
| MapperMergeValidator.validateFieldReferences(fieldMappers, fieldAliasMappers, | |||
| fullPathObjectMappers, fieldTypes); | |||
|
|
|||
| GeoContextMapping.validateGeoContextPaths(indexSettings.getIndexVersionCreated(), fieldMappers, fieldTypes::get); | |||
There was a problem hiding this comment.
Because this method provides a general way to validate context references, I wonder if it makes sense to future-proof a bit and instead reference ContextMapping, and also rename validateGeoContextPaths to validateContextPaths.
There was a problem hiding this comment.
👍 Great catch! Will do.
imotov
added a commit
to imotov/elasticsearch
that referenced
this pull request
Aug 14, 2018
Currently, if geo context is represented by something other than geo_point or an object with lat and lon fields, the parsing of it as a geo context can result in ignoring the context altogether, returning confusing errors such as number_format_exception or trying to parse the number specifying as long-encoded hash code. It would also fail if the geo_point was stored. This commit makes the deprecates creation of geo contexts without correct path fields. And it fixes the issue when geo_point is stored. It is a 6.x version of elastic#32821.
jasontedor
added a commit
to jasontedor/elasticsearch
that referenced
this pull request
Aug 18, 2018
* master: NETWORKING: Make RemoteClusterConn. Lazy Resolve DNS (elastic#32764) [DOCS] Splits the users API documentation into multiple pages (elastic#32825) [DOCS] Splits the token APIs into separate pages (elastic#32865) [DOCS] Creates redirects for role management APIs page Bypassing failing test PainlessDomainSplitIT#testHRDSplit (elastic#32966) TEST: Mute testRetentionPolicyChangeDuringRecovery [DOCS] Fixes more broken links to role management APIs [Docs] Tweaks and fixes to rollup docs [DOCS] Fixes links to role management APIs [ML][TEST] Fix BasicRenormalizationIT after adding multibucket feature [DOCS] Splits the roles API documentation into multiple pages (elastic#32794) [TEST] Run pre 6.4 nodes in non-FIPS JVMs (elastic#32901) Make Geo Context Mapping Parsing More Strict (elastic#32821)
jasontedor
added a commit
that referenced
this pull request
Aug 18, 2018
* elastic/master: (46 commits) NETWORKING: Make RemoteClusterConn. Lazy Resolve DNS (#32764) [DOCS] Splits the users API documentation into multiple pages (#32825) [DOCS] Splits the token APIs into separate pages (#32865) [DOCS] Creates redirects for role management APIs page Bypassing failing test PainlessDomainSplitIT#testHRDSplit (#32966) TEST: Mute testRetentionPolicyChangeDuringRecovery [DOCS] Fixes more broken links to role management APIs [Docs] Tweaks and fixes to rollup docs [DOCS] Fixes links to role management APIs [ML][TEST] Fix BasicRenormalizationIT after adding multibucket feature [DOCS] Splits the roles API documentation into multiple pages (#32794) [TEST] Run pre 6.4 nodes in non-FIPS JVMs (#32901) Make Geo Context Mapping Parsing More Strict (#32821) [ML] fix updating opened jobs scheduled events (#31651) (#32881) Scripted metric aggregations: add deprecation warning and system property to control legacy params (#31597) Tests: Fix timezone conversion in DateTimeUnitTests Enable FIPS140LicenseBootstrapCheck (#32903) Fix InternalAutoDateHistogram reproducible failure (#32723) Remove assertion in testDocStats on deletedDocs counter (#32914) HLRC: Move ML request converters into their own class (#32906) ...
imotov
added a commit
that referenced
this pull request
Aug 20, 2018
Currently, if geo context is represented by something other than geo_point or an object with lat and lon fields, the parsing of it as a geo context can result in ignoring the context altogether, returning confusing errors such as number_format_exception or trying to parse the number specifying as long-encoded hash code. It would also fail if the geo_point was stored. This commit makes the deprecates creation of geo contexts without correct path fields. And it fixes the issue when geo_point is stored. This is a 6.x version of #32821.
jasontedor
pushed a commit
that referenced
this pull request
Aug 21, 2018
Currently, if geo context is represented by something other than geo_point or an object with lat and lon fields, the parsing of it as a geo context can result in ignoring the context altogether, returning confusing errors such as number_format_exception or trying to parse the number specifying as long-encoded hash code. It would also fail if the geo_point was stored. This commit makes the deprecates creation of geo contexts without correct path fields. And it fixes the issue when geo_point is stored. This is a 6.x version of #32821.
jpountz
removed
the
:Analytics/Geo
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently, if geo context is represented by something other than
geo_point or an object with lat and lon fields, the parsing of it
as a geo context can result in ignoring the context altogether,
returning confusing errors such as number_format_exception or trying
to parse the number specifying as long-encoded hash code. It would also
fail if the geo_point was stored.
This commit makes the mapping parsing more strict and will fail during
mapping update or index creation if the geo context doesn't point to
a geo_point field.
Supersedes #32412
Closes #32202