Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SIEM] [Detection Engine] Incorporate large lists to rule execution. #65372

Merged
merged 30 commits into from
May 28, 2020
Merged

[SIEM] [Detection Engine] Incorporate large lists to rule execution. #65372

merged 30 commits into from
May 28, 2020
+770 −288

Commits on May 28, 2020

  1. introduce lists plugin for use by executor

    @dhurley14
    dhurley14 committed May 28, 2020
    Configuration menu
    Copy the full SHA
    c50e966 View commit details
    Browse the repository at this point in the history

  2. adds getListClient function on setup

    @dhurley14
    dhurley14 committed May 28, 2020
    Configuration menu
    Copy the full SHA
    fa86402 View commit details
    Browse the repository at this point in the history

  3. refactors searchAfterBulkCreate to integrate with the lists plugin so… 

    … we only generate signals from events not in the list
    @dhurley14
    dhurley14 committed May 28, 2020
    Configuration menu
    Copy the full SHA
    ecf0191 View commit details
    Browse the repository at this point in the history

  4. fixes type check issues

    @dhurley14
    dhurley14 committed May 28, 2020
    Configuration menu
    Copy the full SHA
    530894d View commit details
    Browse the repository at this point in the history

  5. fixes unit tests, adds field and other parameters for using lists in … 

    …executor.
    @dhurley14
    dhurley14 committed May 28, 2020
    Configuration menu
    Copy the full SHA
    5d68738 View commit details
    Browse the repository at this point in the history

  6. cleaning up types and exports, updates to match new contracts with li… 

    …sts client from master
    @dhurley14
    dhurley14 committed May 28, 2020
    Configuration menu
    Copy the full SHA
    deb2c95 View commit details
    Browse the repository at this point in the history

  7. prior to this commit the refactored while loop was doing more search … 

    …after loops than it needed to and this fixes two bugs in the list filter function where we were returning the wrong count, and we were not accessing the right field on the event
    @dhurley14
    dhurley14 committed May 28, 2020
    Configuration menu
    Copy the full SHA
    e5bd183 View commit details
    Browse the repository at this point in the history

  8. exception lists are optional

    @dhurley14
    dhurley14 committed May 28, 2020
    Configuration menu
    Copy the full SHA
    b729196 View commit details
    Browse the repository at this point in the history

  9. use exceptions list format, this works with given sample query in scr… 

    …ipts
    @dhurley14
    dhurley14 committed May 28, 2020
    Configuration menu
    Copy the full SHA
    878a783 View commit details
    Browse the repository at this point in the history

  10. updates tests and fixes type issues

    @dhurley14
    dhurley14 committed May 28, 2020
    Configuration menu
    Copy the full SHA
    da4833e View commit details
    Browse the repository at this point in the history

  11. updates README doc in detection engine with example for rule with lis… 

    …t exception
    @dhurley14
    dhurley14 committed May 28, 2020
    Configuration menu
    Copy the full SHA
    fe90784 View commit details
    Browse the repository at this point in the history

  12. adds one test and removes commented out code

    @dhurley14
    dhurley14 committed May 28, 2020
    Configuration menu
    Copy the full SHA
    7b7bd81 View commit details
    Browse the repository at this point in the history

  13. fix sample rule json from 30s to 5m

    @dhurley14
    dhurley14 committed May 28, 2020
    Configuration menu
    Copy the full SHA
    2655b84 View commit details
    Browse the repository at this point in the history

  14. fix sample rule json from 30s to 5m

    @dhurley14
    dhurley14 committed May 28, 2020
    Configuration menu
    Copy the full SHA
    b36313e View commit details
    Browse the repository at this point in the history

  15. remove unused import

    @dhurley14
    dhurley14 committed May 28, 2020
    Configuration menu
    Copy the full SHA
    1b0884d View commit details
    Browse the repository at this point in the history

  16. more cleanup

    @dhurley14
    dhurley14 committed May 28, 2020
    Configuration menu
    Copy the full SHA
    e60aa7d View commit details
    Browse the repository at this point in the history

  17. e2e test for prepackaged rules was failing because lists was undefine… 

    …d in the siem plugin and was preventing the registration of the rule alert type. I removed this but once lists is ready for prime time we should consider adding the null check back
    @dhurley14
    dhurley14 committed May 28, 2020
    Configuration menu
    Copy the full SHA
    f2445bf View commit details
    Browse the repository at this point in the history

  18. can't reuse the same env var since the tests are setting the ELASTIC_… 

    …XPACK_SIEM_LISTS_FEATURE env var to true without enabling the lists plugin
    @dhurley14
    dhurley14 committed May 28, 2020
    Configuration menu
    Copy the full SHA
    feb31e1 View commit details
    Browse the repository at this point in the history

  19. fixes from pr review, still needs more TLC

    @dhurley14
    dhurley14 committed May 28, 2020
    Configuration menu
    Copy the full SHA
    87257bd View commit details
    Browse the repository at this point in the history

  20. exports listspluginsetup type from top-level in lists plugin, fixes l… 

    …ogic for empty exceptions list, updates types
    @dhurley14
    dhurley14 committed May 28, 2020
    Configuration menu
    Copy the full SHA
    f3d7684 View commit details
    Browse the repository at this point in the history

  21. utilize type.is to remove as casting, also do null checks and throw a… 

    …n error when exceptionItem is malformed. This will change in the very near future once the new json format for exception lists is incorporated
    @dhurley14
    dhurley14 committed May 28, 2020
    Configuration menu
    Copy the full SHA
    ab8aca3 View commit details
    Browse the repository at this point in the history

  22. fix type issues after merging master into branch

    @dhurley14
    dhurley14 committed May 28, 2020
    Configuration menu
    Copy the full SHA
    204c248 View commit details
    Browse the repository at this point in the history

  23. update mock

    @dhurley14
    dhurley14 committed May 28, 2020
    Configuration menu
    Copy the full SHA
    853a5fb View commit details
    Browse the repository at this point in the history

  24. remove bad null check for ml plugin before registering rule alert typ… 

    …e in siem plugin
    @dhurley14
    dhurley14 committed May 28, 2020
    Configuration menu
    Copy the full SHA
    6b3cd74 View commit details
    Browse the repository at this point in the history

  25. prettier linting

    @dhurley14
    dhurley14 committed May 28, 2020
    Configuration menu
    Copy the full SHA
    f2b235a View commit details
    Browse the repository at this point in the history

  26. adds test for filter events with list

    @dhurley14
    dhurley14 committed May 28, 2020
    Configuration menu
    Copy the full SHA
    54e9861 View commit details
    Browse the repository at this point in the history

  27. pr comments

    @dhurley14
    dhurley14 committed May 28, 2020
    Configuration menu
    Copy the full SHA
    00d47d0 View commit details
    Browse the repository at this point in the history

  28. adds logic for included vs excluded and updates tests

    @dhurley14
    dhurley14 committed May 28, 2020
    Configuration menu
    Copy the full SHA
    6252c57 View commit details
    Browse the repository at this point in the history

  29. update test cases for search after bulk create to default to included… 

    … for exception lists
    @dhurley14
    dhurley14 committed May 28, 2020
    Configuration menu
    Copy the full SHA
    11c30a9 View commit details
    Browse the repository at this point in the history

  30. filter out non-list exception items from the loop

    @dhurley14
    dhurley14 committed May 28, 2020
    Configuration menu
    Copy the full SHA
    d1bf091 View commit details
    Browse the repository at this point in the history