Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SIEM] Final Shimming/Prep for Server NP Migration #56814

Merged
merged 43 commits into from
Feb 14, 2020

Commits on Feb 10, 2020

  1. Route all our server setup through the plugin

    Next we'll be trimming down ServerFacade to the final few dependencies,
    and using our plugin dependencies for the rest.
    rylnd committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    38dbce7 View commit details
    Browse the repository at this point in the history
  2. Clean up server plugin exports

    For now, let's try to simplify our typings by exporting our plugin's
    dependencies from the plugin itself, since we know it already knows
    about them.
    rylnd committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    b8a7000 View commit details
    Browse the repository at this point in the history
  3. Move DE Routes to to conventional location

    I'm throwing the alerting registration in the plugin for now, too.
    rylnd committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    3d23690 View commit details
    Browse the repository at this point in the history
  4. Loosen up our RequestFacade

    Now that we've audited our use of request objects, we can switch to the
    more friendly LegacyRequest that all our utilities are expecting.
    LegacyRequest doesn't have plugins, either, so our only remaining errant
    usage is retrieving clients from it, which we call out explicitly.
    rylnd committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    2e284c5 View commit details
    Browse the repository at this point in the history
  5. Remove uses of 'src' alias

    This just threw an error on startup, so I'm fixing all of them to be
    safe.
    rylnd committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    7a35eac View commit details
    Browse the repository at this point in the history
  6. Fix types of our GraphQL requests

    These come through as new KibanaRequests and not as the LegacyRequest
    that I had incorrectly typed them as.
    
    I'm only caring about the `body` property right now, since that's all we
    really deal with, and in testing it was all that was populated in our
    actual requests, too.
    rylnd committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    5360a87 View commit details
    Browse the repository at this point in the history
  7. Initialize our routes with SetupServices

    We're using the legacy version for now, but ServerFacade will be gone by
    the end of the migration.
    rylnd committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    e41d618 View commit details
    Browse the repository at this point in the history
  8. Swap legacy spaces plugin for NP

    This changes the signature of getIndex, which is used in quite a few
    places. We'll see how bad that looks in the next commit.
    rylnd committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    faf8f77 View commit details
    Browse the repository at this point in the history
  9. Remove unneeded typing

    We're already ignoring another portion of the platform shim due to it
    being incompatibly typed, so we might as well remove this.
    rylnd committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    9418447 View commit details
    Browse the repository at this point in the history
  10. WIP: Converting our DE routes to use consolidated services

    This contains our legacy stuff for now. Eventually, it will only be our
    NP services. This breaks a few tests due to the way createMockServer
    works, but I'll clean that up momentarily.
    rylnd committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    80c5826 View commit details
    Browse the repository at this point in the history
  11. Fix DE routing tests following refactor

    The createMockServer helper does a few things differently:
    
    * returns mocked LegacyServices that can be passed to our route
    factories
    * does not return a server object as we don't need it, except for:
    * returns an inject function that we can use to execute a request
    
    We're casting our services because we're only mocking a subset of what
    LegacySetupServices entails.
    
    Mainly, this allows me to continue moving
    things off of ServerFacade without significant refactoring of tests.
    rylnd committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    b0a7f93 View commit details
    Browse the repository at this point in the history
  12. Fix incompatible request types

    Unfortunately, LegacyRequest does not allow a request's `query` values
    to be anything other than a string or string[]. However, in practice
    they can (and are) objects, booleans, etc.
    
    Our request types (e.g. QueryRequest) are correct, but because service
    functions are LegacyRequest's implementation of `query`, we need to cast
    them lest a type error occur.
    
    For now, the easiest solution is to do this in the request handler:
    intersecting with our RequestFacade (LegacyRequest) allows us to both a)
    pluck our query params off the request and b) pass the request to NP
    services.
    rylnd committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    aa7ae34 View commit details
    Browse the repository at this point in the history
  13. Move our use of encryptedSavedObjects to NP

    We're just retrieving a boolean from it right now, which is also guarded
    against the plugin being unavailable. If this usage becomes more
    widespread, we'll make this available at a higher level, probably in
    redux.
    rylnd committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    b54235d View commit details
    Browse the repository at this point in the history
  14. Use NP elasticsearch client

    * Simplifies our generic type to accept two arguments: params and the
    return value
      * Options is fixed and we were never specifying anything meaningful
      there
    * Updates all DE cluster calls to use callWithRequestFactory
    rylnd committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    3742670 View commit details
    Browse the repository at this point in the history
  15. Update DE mocks with NP elasticsearch

    * createMockServer now returns the callCluster mock, so that you can easily
    mock a client response without needing to know the details of how we
    define that function
    rylnd committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    8ccaf4b View commit details
    Browse the repository at this point in the history
  16. Remove savedObjects dependency from our legacy dependencies

    This was added during a refactor, but we were never actually using this
    code. We always retrieve the client from the request via
    getSavedObjectsClient.
    
    I think that the NP client has a slightly different interface, so we're
    going to create a helper to retrieve it from the request, same as we do
    with the elastic client.
    
    In the future, both of these will be available on the request context,
    and we can remove the helpers entirely.
    rylnd committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    98d56b8 View commit details
    Browse the repository at this point in the history
  17. WIP: Convert services to stateful object

    In trying to migrate over the savedObjectsClient, I realized that it is
    not available during setup in the same way that the ES client is.
    
    Since our routes need pieces from both setup and start phases, I've
    added a Services class to accumulate/transform these services and expose
    scoped clients when given a legacy request.
    
    In New Platform, these clients will be available upon the request
    context and we should be able to remove getScopedServicesFactory for our
    routes. A subset of Services' functionality may still be useful, we'll
    see.
    rylnd committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    00d089d View commit details
    Browse the repository at this point in the history
  18. WIP: Converting routes to use Services factory

    I decided that config shouldn't live in here, as this is only
    client-related stuff. Probably going to rename this ClientsService.
    
    Things are still very much broken.
    rylnd committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    35bfc0b View commit details
    Browse the repository at this point in the history
  19. WIP: Qualifying our Service to ClientsService

    This gets us client-related services (ES, SavedObjects, Alerts), but it
    is independent of any configuration, which is gonna be another service.
    rylnd committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    7e7cd30 View commit details
    Browse the repository at this point in the history
  20. Fix types on getIndex function

    This is a weird helper, I'm not really sure where it should go.
    rylnd committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    c97f190 View commit details
    Browse the repository at this point in the history
  21. Our ClientsService is a clients ... service

    Return clients, as this is closer to what we'll get in the request
    context.
    rylnd committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    101993f View commit details
    Browse the repository at this point in the history
  22. Clean up our server types

    * Declare legacy types at top-level file
    * Don't re-export from the plugin solely for convenience, that's a
    slippery slope straight to circular dependencies
    * Remove RequestFacade as it was a facade for LegacyRequest
    * Rename ServerFacade/LegacySetupServices to just LegacyServices
    rylnd committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    f7bb2fb View commit details
    Browse the repository at this point in the history
  23. Refactor mocks for new architecture

    * Separates config, server, and client mocks, as they're now independent
    in our system, route-wise.
    * gets one test working, the rest will follow.
    rylnd committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    2502698 View commit details
    Browse the repository at this point in the history
  24. Simplify our routing mocks

    * Adds mock for our new clients service
    * Greatly simplifies both server and mock configs
    * Renames factory method of client service
    rylnd committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    36398bc View commit details
    Browse the repository at this point in the history
  25. Loosen graphQL endpoint validations

    These work fine in production, but it's graphQL so we don't really need
    the additional validation of these endpoints, and we weren't leveraging
    these types anywhere in Typescript land.
    
    Additionally, these restrictive validations prevent the initial
    introspection calls done by graphiQL to get schema information, and
    without schemae graphiql wasn't very helpful. This is a dev-only
    problem, but that's the audience of graphiql.
    rylnd committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    1e6f106 View commit details
    Browse the repository at this point in the history
  26. Remove unused graphql endpoint

    This was only registered in dev mode; I thought that it was needed by
    graphiql. However, after digging further I realized that graphiQL also
    only makes POST calls to our real graphQL endpoint, so this route is
    unnecessary.
    rylnd committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    bcff5e1 View commit details
    Browse the repository at this point in the history
  27. Reduce our dependence on PluginInitializerContext

    After a little more introspection I realized our FrameworkAdapter
    doesn't need the kibana version. It was only used in order to make a dev
    request via (graphiql), but even that can be performed with a simpler
    xsrf header.
    
    This meant that we really only wanted to know whether we're in
    production or not, so instead we pass that simple boolean to the
    constructor.
    rylnd committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    f84a3d7 View commit details
    Browse the repository at this point in the history
  28. Fix FrameworkAdapter type

    We no longer need this property.
    rylnd committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    930b89a View commit details
    Browse the repository at this point in the history
  29. Update detections route tests

    Uses the new routes interfaces, and our corresponding new mocks.
    rylnd committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    c3c67f1 View commit details
    Browse the repository at this point in the history
  30. Remove unnecessary null checks

    Our savedObjectsClient is always going to be there.
    rylnd committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    79171ff View commit details
    Browse the repository at this point in the history
  31. Remove unused type

    YAGNI
    rylnd committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    156dd77 View commit details
    Browse the repository at this point in the history
  32. Remove unused savedObjects client

    Turns out we were only destructuring this client for the null check.
    rylnd committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    e309001 View commit details
    Browse the repository at this point in the history
  33. Handle case where spaces is disabled

    We already null-coalesce properly in the clients service, but this
    property access was missed.
    rylnd committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    0816e62 View commit details
    Browse the repository at this point in the history
  34. Configuration menu
    Copy the full SHA
    dfd81cd View commit details
    Browse the repository at this point in the history
  35. Remove unnecessary casting of our alerts client mock

    I think that this was the result of us importing the wrong AlertsClient
    type, or perhaps the types were out of sync. Regardless, they work now.
    rylnd committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    11f3581 View commit details
    Browse the repository at this point in the history
  36. Return the 'default' space even when spaces are disabled

    This will allow users with spaces disabled to enable spaces without
    losing data. The tradeoff is that they may be surprised when signals
    don't exist within their configured xpack.siem.signalsIndex.
    rylnd committed Feb 10, 2020
    Configuration menu
    Copy the full SHA
    0e98dae View commit details
    Browse the repository at this point in the history

Commits on Feb 12, 2020

  1. Merge branch 'master' into final_backend_prep

     Conflicts:
    	x-pack/legacy/plugins/siem/server/kibana.index.ts
    	x-pack/legacy/plugins/siem/server/lib/detection_engine/routes/index/create_index_route.ts
    	x-pack/legacy/plugins/siem/server/lib/detection_engine/routes/index/delete_index_route.ts
    	x-pack/legacy/plugins/siem/server/lib/detection_engine/routes/rules/add_prepackaged_rules_route.test.ts
    	x-pack/legacy/plugins/siem/server/lib/detection_engine/routes/rules/add_prepackaged_rules_route.ts
    	x-pack/legacy/plugins/siem/server/lib/detection_engine/routes/rules/create_rules_route.ts
    	x-pack/legacy/plugins/siem/server/lib/detection_engine/routes/rules/delete_rules_bulk_route.ts
    	x-pack/legacy/plugins/siem/server/lib/detection_engine/routes/rules/find_rules_route.ts
    	x-pack/legacy/plugins/siem/server/lib/detection_engine/routes/rules/update_rules_bulk_route.ts
    	x-pack/legacy/plugins/siem/server/lib/detection_engine/routes/rules/update_rules_route.ts
    	x-pack/legacy/plugins/siem/server/lib/detection_engine/rules/types.ts
    	x-pack/legacy/plugins/siem/server/lib/detection_engine/types.ts
    rylnd committed Feb 12, 2020
    Configuration menu
    Copy the full SHA
    8dc5622 View commit details
    Browse the repository at this point in the history

Commits on Feb 13, 2020

  1. Merge branch 'master' into final_backend_prep

     Conflicts:
    	x-pack/legacy/plugins/siem/server/lib/detection_engine/routes/utils.test.ts
    	x-pack/legacy/plugins/siem/server/lib/detection_engine/routes/utils.ts
    rylnd committed Feb 13, 2020
    Configuration menu
    Copy the full SHA
    4e9f8d8 View commit details
    Browse the repository at this point in the history
  2. Account for spaces being disabled in ClientsService

    * Updates types to reflect that spaces may be unavailable
    * Adds a test for getSpaceId's behavior when spaces are disabled
    rylnd committed Feb 13, 2020
    Configuration menu
    Copy the full SHA
    f53e6f7 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    cb5d729 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    668702b View commit details
    Browse the repository at this point in the history
  5. Fix false positives in query signals routes tests

    * Refactors mock expectations so that they're actually evaluated; they
    can't go within a mockImplementation call as it's evaluated in the wrong
    scope.
    * Fixes duplicated test to use the proper assertions
    rylnd committed Feb 13, 2020
    Configuration menu
    Copy the full SHA
    56fc3f5 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    7162528 View commit details
    Browse the repository at this point in the history