[ftr] pass password for UI login + improve login/logout steps with proper wait logic

[dmlemeshko]

Summary

Hopefully

closes #167104
closes #167130
closes #167100
closes #167013
closes #166964

Fixing a few issues with login/logout:

1. Failed to login in "before" hook

My theory is that we are loading /login route too soon while log out was not completed yet.
When we navigate to https://localhost:5620/logout there are multiple url re-directions with final page being Cloud login form. This PR makes sure we wait for this form to be displayed + 2500 ms extra to avoid "immediate" /login navigation

2. Failed login on MKI:
   Updating login via UI for serverless to pass password valid for deployment: currently FTR uses changeme for both Kibana CI & MKI.

3. ES activate user profile call returning 500
   We saw some login failures that are preceded with the following logs:

[00:03:27]           │ debg Find.clickByCssSelector('[data-test-subj="loginSubmit"]') with timeout=10000
[00:03:27]           │ debg Find.findByCssSelector('[data-test-subj="loginSubmit"]') with timeout=10000
[00:03:27]           │ debg Find.waitForDeletedByCssSelector('.kibanaWelcomeLogo') with timeout=10000
[00:03:27]           │ proc [kibana] [2023-09-19T07:08:26.126+00:00][INFO ][plugins.security.routes] Logging in with provider "basic" (basic)
[00:03:27]           │ info [o.e.x.s.s.SecurityIndexManager] [ftr] security index does not exist, creating [.security-profile-8] with alias [.security-profile]
[00:03:27]           │ proc [kibana] [2023-09-19T07:08:26.140+00:00][ERROR][plugins.security.user-profile] Failed to activate user profile: {"error":{"root_cause":[{"type":"validation_exception","reason":"Validation Failed: 1: this action would add [1] shards, but this cluster currently has [27]/[27] maximum normal shards open;"}],"type":"validation_exception","reason":"Validation Failed: 1: this action would add [1] shards, but this cluster currently has [27]/[27] maximum normal shards open;"},"status":400}.
[00:03:27]           │ proc [kibana] [2023-09-19T07:08:26.140+00:00][ERROR][http] 500 Server Error
[00:03:27]           │ warn browser[SEVERE] http://localhost:5620/internal/security/login - Failed to load resource: the server responded with a status of 500 (Internal Server Error)

User activation happens during POST internal/security/login call to Kibana server. The only improvement that we can do from FTR perspective is to call this end-point via API to makes sure user is activated and only after proceed with UI login.
While working on issue #4 and talking to @jeramysoucy I believe retrying login via UI will work here as well. We are checking if we are still on login page (similar to incorrect password login), waiting 2500 ms and pressing login button again.

4. Failed to login with Kibana reporting UNEXPECTED_SESSION_ERROR and been re-directed to Cloud login page

proc [kibana] [2023-09-25T11:35:12.794+00:00][INFO ][plugins.security.authentication] Authentication attempt failed: UNEXPECTED_SESSION_ERROR

Temporary solution is to retry login from scratch (navigation to Kibana login page & re-login )

Flaky-test-runner for functional obtl tests 50x https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3215

This PR is not fixing random 401 response when user navigates to some apps with direct url

[sabarasaba]

Thanks for having a go at this @dmlemeshko! It has been puzzling us why these were failing so randomly. I noticed in your flaky test runner run that some of the tests for test_suites/common/index_management are still failing with either:

• [data-test-subj="userMenuButton"] is not displayed
• Waiting for element to be located By(css selector, [data-test-subj="kibanaChrome"])

Are these errors also something that can be caused by the problem you defined in the PR summary?

[dmlemeshko]

Hey @sabarasaba

The problem you described is unfortunately still not fixed and I will revert my last changes related to navigateToApp.
Sharing some light on the issue:

After login many tests load a direct url to the Kibana app and sometimes these requests are resulting in a 401 response, and instead of Kibana app user ends up stuck at https://localhost:5620/api/security/saml/callback. For some reason https://localhost:5620/app/management/data/index_management is one of those flaky paths

[00:01:33]           | debg userMenuButton is found, logged in passed
[00:01:33]           │ debg Logged in successfully
[00:01:33]           │ debg navigating to indexManagement url: https://localhost:5620/app/management/data/index_management
[00:01:33]           │ debg navigate to: https://localhost:5620/app/management/data/index_management
[00:01:33]           │ debg browser[WARNING] security - Error with Permissions-Policy header: Unrecognized feature: 'web-share'.
[00:01:33]           │ warn browser[SEVERE] https://localhost:5620/bootstrap.js - Failed to load resource: the server responded with a status of 401 (Unauthorized)
[00:01:33]           │ debg browser[INFO] https://localhost:5620/app/management/data/index_management?_t=1695726310497 285 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.

You can see that bootstrap.js failed with 401, right after user logged in successfully and loaded https://localhost:5620/app/management/data/index_management

You can follow elastic/kibana-team/issues/629 for more details

[pheyos]

LGTM

[dmlemeshko]

@elasticmachine merge upstream

[kibana-ci]

💔 Build Failed

• Buildkite Build
• Commit: 1dace28
• Interpreting CI Failures

Failed CI Steps

• Check Types Commit Diff

Metrics [docs]

✅ unchanged

History

• 💔 Build #163113 failed 76ac2fc
• 💔 Build #163066 failed 96dd1ac
• 💔 Build #163037 failed 6a02860
• 💔 Build #163015 failed ba20b20
• 💔 Build #162876 failed 1cf92ce

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @dmlemeshko