-
Notifications
You must be signed in to change notification settings - Fork 8.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[ftr] implement support for accessing ES through CCS #126547
Conversation
94d23e8
to
a02cfb2
Compare
a02cfb2
to
d6f76fa
Compare
d6f76fa
to
0498ec0
Compare
Pinging @elastic/kibana-operations (Team:Operations) |
Pinging @elastic/kibana-qa (Team:QA) |
@elasticmachine merge upstream |
Yes set like this:
The curl command for those URLs comes back successful.
|
@spalger your fix worked for the superuser security issue. Thanks! I only ran the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, tested FTR changes against cloud CCS setup.
@spalger can you explain how I would (in a test) get the test_user created with the correct roles on the remote cluster? The existing functional_ccs tests copied to x-pack and inheriting x-pack/test/functional/config.js won't pass for me locally unless it also created test_user on the remote cluster. I'm not sure we can use the existing test_user service if it's hitting the Kibana api. We need to use the remote Elasticsearch cluster to create the user. |
@LeeDr My understanding of https://www.elastic.co/guide/en/elasticsearch/reference/current/remote-clusters-privileges.html#remote-clusters-privileges-ccs was that we just needed to have a known role name in sync between the remote cluster and the local user, so the testUser would still be created on the local cluster but it would need to get a role name that is in place on the remote cluster and gives it the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM - Spencer is correct that we don't need the test_user on the remote cluster. Only the role including view_index_metadata
. Works great!
💚 Build SucceededMetrics [docs]Public APIs missing comments
History
To update your PR or re-run it, just comment with: |
💔 All backports failed
Manual backportTo create the backport manually run:
Questions ?Please refer to the Backport tool documentation |
…ed-unexpectedly-error * 'main' of github.com:elastic/kibana: (46 commits) Fix copy and pasted renderer user_name test (elastic#126663) [Gauge] Vis editors gauge legacy percent mode. (elastic#126318) Remove all cases related code from timelines (elastic#127003) Hide Enterprise search panel when no nodes are present (elastic#127100) [Lens] Fixed flakiness on runtime fields' appearance on the list (elastic#126945) [Security Solution][Lists] - Add missing privileges callout to exception lists page (elastic#126874) [Security Solution][Lists] - Updates exception flyout edit error messages (elastic#126875) [Security Solution][Rules] - Remove rule selection for read only users (elastic#126827) Fix session cleanup test (elastic#126966) [ftr] implement support for accessing ES through CCS (elastic#126547) [type-summarizer] always use normalized paths, fix windows compat (elastic#127055) Revert "[ci] Configure hourly pipeline for a small spot instance trial (elastic#126824)" Revert "[CI] Expand spot instance trial a bit (elastic#126928)" [Alerting] Adding functional tests for alerting and actions telemetry (elastic#126528) [Telemetry] Check permissions when requesting telemetry (elastic#126238) Don't submit empty seed_urls or sitemap_urls when making a partial crawl request (elastic#126972) Remove License Requirement for Enterprise Search App Search Meta Engines (elastic#127046) [ML] Adding data recognizer module config cache (elastic#126338) skip flaky suite (elastic#126027) [Reporting] Improve error logging for rescheduled jobs (elastic#126737) ... # Conflicts: # x-pack/plugins/reporting/server/core.ts # x-pack/plugins/reporting/server/lib/tasks/execute_report.ts
…re-browser-errors * 'main' of github.com:elastic/kibana: (46 commits) Fix copy and pasted renderer user_name test (elastic#126663) [Gauge] Vis editors gauge legacy percent mode. (elastic#126318) Remove all cases related code from timelines (elastic#127003) Hide Enterprise search panel when no nodes are present (elastic#127100) [Lens] Fixed flakiness on runtime fields' appearance on the list (elastic#126945) [Security Solution][Lists] - Add missing privileges callout to exception lists page (elastic#126874) [Security Solution][Lists] - Updates exception flyout edit error messages (elastic#126875) [Security Solution][Rules] - Remove rule selection for read only users (elastic#126827) Fix session cleanup test (elastic#126966) [ftr] implement support for accessing ES through CCS (elastic#126547) [type-summarizer] always use normalized paths, fix windows compat (elastic#127055) Revert "[ci] Configure hourly pipeline for a small spot instance trial (elastic#126824)" Revert "[CI] Expand spot instance trial a bit (elastic#126928)" [Alerting] Adding functional tests for alerting and actions telemetry (elastic#126528) [Telemetry] Check permissions when requesting telemetry (elastic#126238) Don't submit empty seed_urls or sitemap_urls when making a partial crawl request (elastic#126972) Remove License Requirement for Enterprise Search App Search Meta Engines (elastic#127046) [ML] Adding data recognizer module config cache (elastic#126338) skip flaky suite (elastic#126027) [Reporting] Improve error logging for rescheduled jobs (elastic#126737) ... # Conflicts: # x-pack/plugins/reporting/server/lib/tasks/execute_report.ts
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Friendly reminder: Looks like this PR hasn’t been backported yet. |
Backport blocked by #124586, going to skip it for now. If you get the backports done and would like me to backport this please let me know @cuff-links |
Will do. |
Closes #124865
In order to support testing CCS compatibility we have decided to extend the FTR config schema to support
esTestCluster.ccs
which, when enabled, changes the way that we start ES to start two single-node clusters:1.
ftr-remote
This cluster is started first, it listens to HTTP requests at
esTestCluster.ccs.remoteClusterUrl
, and transport requests on a port determined at runtime using theget-port
module. TheremoteEs
andremoteEsArchiver
services are setup to communicate directly with this node for loading data into and managing the remote cluster. Roles on this cluster can be setup with theesTestCluster.remoteRoles
config.2.
ftr-local
This cluster is initialized second, it is the cluster that Kibana and all the default services will talk to, including
es
andesArchiver
. This cluster also has a single remote cluster configured at startup,ftr-remote
which can be used to query theftr-remote
cluster.In the
test/functional_ccs/config.ts
file we setup theccs_remote_search
role on the remote cluster, and then also set this as a default role for all users created on via thetestUser
service. When callingtestUser.setRoles()
people will need to define this role if they want that user to have CCS permissions on the remote cluster. We could change the logic to be out-out, but I'd prefer to make it explicit when we callsetRoles()
that the list of roles provided will be the roles that user has.Additionally, in order to support cloud testing this config supports defining the entire
esTestCluster.ccs.remoteClusterUrl
via theREMOTE_CLUSTER_URL
environment variable, which could point to a completely different machine.