EQL: Remove "wildcard" function #76099

bpintea · 2021-08-04T14:54:52Z

This removes "wildcard" as an available function in EQL. This has
already been replace with "like" and "regex" embedded synthax (and
respective case insensitive variants).

Relates #71906.

This removes "wildcard" as an available function in EQL. This has already been replace with "like" and "regex" embedded synthax (and respective case insensitive variants).

elasticmachine · 2021-08-04T15:41:48Z

Pinging @elastic/es-ql (Team:QL)

rw-access

LGTM.
My only note is that I'm not sure where wildcard should be deleted and where it should be changed to like.

rw-access · 2021-08-04T15:42:49Z

x-pack/plugin/eql/qa/common/src/main/resources/additional_test_queries.toml

-[[queries]]
-name = "wildcardFunctionWildcardPattern"
-query = '''
-file where wildcard(file_name, "winini*.exe", "lsass.*") and opcode == 2


should this be replaced with like?

There's already a like test for this (likeMultipleArgWithPattern).

rw-access · 2021-08-04T15:45:18Z

x-pack/plugin/eql/src/test/resources/test_string_functions.toml

@@ -411,73 +411,3 @@ description = "Test the `substring` function when the case already matches"
    [[substring.fold.tests]]
    expression = '''substring("hello world", -5, -1)'''
    expected = "worl"
-
-[wildcard]


should this one be replaced with like? is this file even used?

Yes, file still in use.

Luegg

Lgtm 👍

bpintea · 2021-08-04T17:37:05Z

My only note is that I'm not sure where wildcard should be deleted and where it should be changed to like.

I've tried to preserve the wildcard-now-like tests where they're part of a more complex suite (like those in queries.toml) and removed them where wildcard itself was tested, since much of these are duplicated by like testing.

costin

Assuming the removed tests already have equivalent for like the only comment I have is around fully removing the Wildcard class which is not needed and the substitute construct can be created directly by the parser.

costin · 2021-08-05T07:38:18Z

...k/plugin/eql/src/test/java/org/elasticsearch/xpack/eql/planner/QueryTranslatorFailTests.java

@@ -205,35 +205,4 @@ public void testStringContainsWrongParams() {
        assertEquals("1:15: second argument of [stringContains(process_name, 1)] must be [string], found value [1] type [integer]",
                error("process where stringContains(process_name, 1)"));
    }
-
-    public void testWildcardNotEnoughArguments() {


Please convert these to like if there aren't tests that already do that.

These tests become now syntactic failures. One case could however be indeed converted (ex testWildcardWithNumericField).

As long as we have equivalent semantical tests, I'm 👍 . Note that wildcard tests that rely on the syntax can be changed to like/like~ hence my point of replacing removed tests (or checking) there are semantical equivalents to them.

costin · 2021-08-05T07:39:52Z

...ql/src/main/java/org/elasticsearch/xpack/eql/expression/function/scalar/string/Wildcard.java

@@ -31,7 +31,7 @@

 /**
 * EQL wildcard function. Matches the form:
- *     wildcard(field, "*wildcard*pattern*", ...)
+ *     field like ("*wildcard*pattern*", ...)
 */
 public class Wildcard extends BaseSurrogateFunction {


No need to keep this class around.
It's used inside the parser ExpressionBuilder#visitOperatorExpressionDefault however its only role is to call makeSubstitute which is similar to ExpressionBuilder#combineExpressions. See the handling of EqlBaseParser.IN_INSENSITIVE: in visitOperatorExpressionDefault.

Thanks. Class removed.

- remove Wildcard class; - recover one failed translation test.

costin

LGTM

bpintea · 2021-08-16T09:53:49Z

@elasticmachine run elasticsearch-ci/bwc

bpintea · 2021-08-16T14:30:43Z

@elasticmachine update branch

* master: (868 commits) Query API key - Rest spec and yaml tests (elastic#76238) Delay shard reassignment from nodes which are known to be restarting (elastic#75606) Reenable bwc tests for elastic#76475 (elastic#76576) Set version to 7.15 in BWC code (elastic#76577) Don't remove warning headers on all failure (elastic#76434) Disable bwc tests for elastic#76475 (elastic#76541) Re-enable bwc tests (elastic#76567) Keep track of data recovered from snapshots in RecoveryState (elastic#76499) [Transform] Align transform checkpoint range with date_histogram interval for better performance (elastic#74004) EQL: Remove "wildcard" function (elastic#76099) Fix 'accept' and 'content_type' fields for search_mvt API Add persistent licensed feature tracking (elastic#76476) Add system data streams to feature state snapshots (elastic#75902) fix the error message for instance methods that don't exist (elastic#76512) ILM: Add validation of the number_of_shards parameter in Shrink Action of ILM (elastic#74219) remove dashboard only reserved role (elastic#76507) Fix Stack Overflow in UnassignedInfo in Corner Case (elastic#76480) Add (Extended)KeyUsage KeyUsage, CipherSuite & Protocol to SSL diagnostics (elastic#65634) Add recovery from snapshot to tests (elastic#76535) Reenable BwC Tests after elastic#76532 (elastic#76534) ...

Remove wildcard function

783a1d2

This removes "wildcard" as an available function in EQL. This has already been replace with "like" and "regex" embedded synthax (and respective case insensitive variants).

bpintea added >deprecation v8.0.0 :Analytics/EQL EQL querying labels Aug 4, 2021

bpintea requested review from rw-access, Luegg, costin and astefan August 4, 2021 15:41

bpintea marked this pull request as ready for review August 4, 2021 15:41

elasticmachine added the Team:QL (Deprecated) Meta label for query languages team label Aug 4, 2021

rw-access approved these changes Aug 4, 2021

View reviewed changes

Luegg approved these changes Aug 4, 2021

View reviewed changes

costin requested changes Aug 5, 2021

View reviewed changes

bpintea added 2 commits August 5, 2021 18:46

Address review feedback

97fde83

- remove Wildcard class; - recover one failed translation test.

Merge branch 'master' into fix/rm_wildcard

5a22ca6

costin approved these changes Aug 9, 2021

View reviewed changes

Merge branch 'master' into fix/rm_wildcard

b8a65f5

Merge branch 'master' into fix/rm_wildcard

917a835

bpintea merged commit a388f70 into elastic:master Aug 16, 2021

bpintea deleted the fix/rm_wildcard branch August 16, 2021 15:48

jrodewig added >breaking v8.0.0-alpha1 and removed v8.0.0 labels Sep 10, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

EQL: Remove "wildcard" function #76099

EQL: Remove "wildcard" function #76099

bpintea commented Aug 4, 2021

elasticmachine commented Aug 4, 2021

rw-access left a comment

rw-access Aug 4, 2021

bpintea Aug 4, 2021

rw-access Aug 4, 2021

bpintea Aug 4, 2021

Luegg left a comment

bpintea commented Aug 4, 2021

costin left a comment

costin Aug 5, 2021

bpintea Aug 8, 2021

costin Aug 9, 2021

costin Aug 5, 2021

bpintea Aug 8, 2021

costin left a comment

bpintea commented Aug 16, 2021

bpintea commented Aug 16, 2021

EQL: Remove "wildcard" function #76099

EQL: Remove "wildcard" function #76099

Conversation

bpintea commented Aug 4, 2021

elasticmachine commented Aug 4, 2021

rw-access left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Luegg left a comment

Choose a reason for hiding this comment

bpintea commented Aug 4, 2021

costin left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

costin left a comment

Choose a reason for hiding this comment

bpintea commented Aug 16, 2021

bpintea commented Aug 16, 2021