Failing test: Chrome X-Pack UI Functional Tests.x-pack/test/functional/apps/uptime/synthetics_integration·ts - Uptime app with generated data When on the Synthetics Integration Policy Create Page displays custom UI "before all" hook for "should display policy view" #116522

kibanamachine · 2021-10-27T22:57:29Z

A test failed on a tracked branch

Error: retry.try timeout: Error: Synthetics package was not in response from /api/fleet/epm/packages
    at /opt/local-ssd/buildkite/builds/kb-cigroup-6-4026b0466283d460/elastic/kibana-hourly/kibana/x-pack/test/functional/services/uptime/synthetics_package.ts:64:23
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at runAttempt (/opt/local-ssd/buildkite/builds/kb-cigroup-6-4026b0466283d460/elastic/kibana-hourly/kibana/test/common/services/retry/retry_for_success.ts:29:15)
    at retryForSuccess (/opt/local-ssd/buildkite/builds/kb-cigroup-6-4026b0466283d460/elastic/kibana-hourly/kibana/test/common/services/retry/retry_for_success.ts:68:21)
    at RetryService.try (/opt/local-ssd/buildkite/builds/kb-cigroup-6-4026b0466283d460/elastic/kibana-hourly/kibana/test/common/services/retry/retry.ts:31:12)
    at Object.getSyntheticsPackageVersion (test/functional/services/uptime/synthetics_package.ts:84:33)
    at Context.<anonymous> (test/functional/apps/uptime/synthetics_integration.ts:157:25)
    at Object.apply (/opt/local-ssd/buildkite/builds/kb-cigroup-6-4026b0466283d460/elastic/kibana-hourly/kibana/node_modules/@kbn/test/target_node/functional_test_runner/lib/mocha/wrap_function.js:87:16)
    at onFailure (/opt/local-ssd/buildkite/builds/kb-cigroup-6-4026b0466283d460/elastic/kibana-hourly/kibana/test/common/services/retry/retry_for_success.ts:17:9)
    at retryForSuccess (/opt/local-ssd/buildkite/builds/kb-cigroup-6-4026b0466283d460/elastic/kibana-hourly/kibana/test/common/services/retry/retry_for_success.ts:59:13)
    at RetryService.try (/opt/local-ssd/buildkite/builds/kb-cigroup-6-4026b0466283d460/elastic/kibana-hourly/kibana/test/common/services/retry/retry.ts:31:12)
    at Object.getSyntheticsPackageVersion (test/functional/services/uptime/synthetics_package.ts:84:33)
    at Context.<anonymous> (test/functional/apps/uptime/synthetics_integration.ts:157:25)
    at Object.apply (/opt/local-ssd/buildkite/builds/kb-cigroup-6-4026b0466283d460/elastic/kibana-hourly/kibana/node_modules/@kbn/test/target_node/functional_test_runner/lib/mocha/wrap_function.js:87:16)

First failure: CI Build - 8.0

The text was updated successfully, but these errors were encountered:

kibanamachine · 2021-10-27T23:51:39Z

New failure: CI Build - 8.0

elasticmachine · 2021-10-28T02:12:22Z

Pinging @elastic/uptime (Team:uptime)

joshdover · 2021-10-28T10:16:37Z

The test is here:

kibana/x-pack/test/functional/services/uptime/synthetics_package.ts

Lines 47 to 67 in acf0426

    
           // Currently (as of 2020-june) the package registry used in CI is the public one and 
        
           // at times it encounters network connection issues. We use `retry.try` below to see if 
        
           // subsequent requests get through. 
        
           apiRequest = retry.try(() => { 
        
             return supertest 
        
               .get(INGEST_API_EPM_PACKAGES) 
        
               .set('kbn-xsrf', 'xxx') 
        
               .expect(200) 
        
               .catch((error) => { 
        
                 return logSupertestApiErrorAndThrow(`Unable to retrieve packages via Ingest!`, error); 
        
               }) 
        
               .then((response: { body: GetPackagesResponse }) => { 
        
                 const { body } = response; 
        
                 const syntheticsPackageInfo = body.response.find( 
        
                   (epmPackage) => epmPackage.name === 'synthetics' 
        
                 ); 
        
                 if (!syntheticsPackageInfo) { 
        
                   throw new Error( 
        
                     `Synthetics package was not in response from ${INGEST_API_EPM_PACKAGES}` 
        
                   ); 
        
                 }

I think the comment at the top indicates the issue. Instead of using the production EPR, we should set these tests up to use a local docker image of EPR just like we do for Fleet's API integration tests.

This can be done by:

Setting the dockerServers option in the FTR config like this:

kibana/x-pack/test/fleet_api_integration/config.ts

Lines 43 to 52 in acf0426

    
           dockerServers: defineDockerServersConfig({ 
        
             registry: { 
        
               enabled: !!registryPort, 
        
               image: dockerImage, 
        
               portInContainer: 8080, 
        
               port: registryPort, 
        
               args: dockerArgs, 
        
               waitForLogLine: 'package manifests loaded', 
        
             }, 
        
           }),

Adding the --xpack.fleet.registryUrl CLI flag in the FTR config like this:

kibana/x-pack/test/fleet_api_integration/config.ts

Line 65 in acf0426

...(registryPort ? [`--xpack.fleet.registryUrl=http://localhost:${registryPort}`] : []),
(optional) Add a hook like this one to tests to skip them when the Docker container is not configured:

kibana/x-pack/test/fleet_api_integration/apis/epm/get.ts

Line 40 in acf0426

skipIfNoDockerRegistry(providerContext);

kibanamachine · 2021-10-28T11:49:12Z

New failure: CI Build - 8.0

kibanamachine · 2021-10-28T12:43:15Z

New failure: CI Build - 8.0

kibanamachine · 2021-10-28T16:03:23Z

New failure: CI Build - 8.0

kibanamachine · 2021-10-28T16:20:23Z

New failure: CI Build - 8.0

kibanamachine · 2021-10-28T16:57:04Z

New failure: CI Build - 8.0

kibanamachine · 2021-10-28T17:14:23Z

New failure: CI Build - 8.0

jbudz · 2021-10-28T17:16:47Z

skipped

8.0: 9e2bda7

kibanamachine · 2021-10-28T17:18:43Z

New failure: CI Build - 8.0

kibanamachine · 2021-10-28T18:07:49Z

New failure: CI Build - 8.0

kibanamachine · 2021-10-28T18:16:58Z

New failure: CI Build - 8.0

kibanamachine · 2021-10-28T19:10:57Z

New failure: CI Build - 8.0

lucasfcosta · 2021-11-18T15:14:35Z

@dominiqueclarke this issue seems to be in the Needs Review column, and I couldn't see the changes pointed by @joshdover in the other PR. However, you mentioned your PR solves this issue.

Can you confirm or let me know if I need to do further changes to address this flaky test?

dominiqueclarke · 2021-11-23T12:42:13Z

@paulb-elastic The tests are now unskipped, but this ticket relates to an enhancement to our current tests. Moving to 8.0.0.

lucasfcosta · 2021-12-21T14:45:57Z

As we previously discussed, addressing these flaky tests depend on having a local EPR.

This issue seems to have been somewhat superseded by the initiative described in #116591, so I added a comment to double check whether the Kibana Operations/Fleet teams expect us to implement the local EPR ourselves or whether they'll do it across the board for consistency/technical/other reasons.

dominiqueclarke · 2022-01-25T15:11:09Z

Closed by #117736

kibanamachine · 2022-02-16T21:16:28Z

New failure: CI Build - 7.15

kibanamachine · 2022-02-16T21:28:15Z

New failure: CI Build - 7.15

dominiqueclarke · 2022-02-23T17:30:14Z

@paulb-elastic This should be resolved by #125127. Let's see if another failure pops up. Don't think it's worth investigating at this time.

dominiqueclarke · 2022-02-23T18:42:34Z

I think I spoke too soon on this one. I initially thought the new failures were from 8.2.0, but now that I see they are in 7.15, we should investigate. This is likely do to this change in package registry: elastic/package-registry@4a270e4

joshdover · 2022-02-24T12:08:21Z

@dominiqueclarke Since we're no longer maintaining the 7.15 branch I think we can just close this now.

dominiqueclarke · 2022-02-25T15:36:19Z

Closing due to 7.15 no longer being maintained.

kibanamachine · 2022-03-03T16:02:25Z

New failure: CI Build - 7.15

kibanamachine · 2022-03-03T16:14:41Z

New failure: CI Build - 7.15

kibanamachine added the failed-test A test failure on a tracked branch, potentially flaky-test label Oct 27, 2021

botelastic bot added the needs-team Issues missing a team label label Oct 27, 2021

mistic added the Team:Uptime - DEPRECATED Synthetics & RUM sub-team of Application Observability label Oct 28, 2021

botelastic bot removed the needs-team Issues missing a team label label Oct 28, 2021

jbudz added a commit that referenced this issue Oct 28, 2021

skip failing suite. #116522

9e2bda7

jen-huang mentioned this issue Oct 28, 2021

[Fleet] Fix branch conditions to check against main and add skip version check config option #116530

Merged

dominiqueclarke mentioned this issue Nov 15, 2021

[Uptime] adjust Elastic Synthetics integration functional tests #118163

Merged

lucasfcosta self-assigned this Nov 18, 2021

dominiqueclarke assigned dominiqueclarke and unassigned lucasfcosta Jan 11, 2022

dominiqueclarke closed this as completed Jan 25, 2022

kibanamachine reopened this Feb 16, 2022

This was referenced Feb 23, 2022

Synthetics functional tests - Query for experimental packages #126281

Merged

Synthetics functional tests - query for experimental packages #126282

Closed

dominiqueclarke closed this as completed Feb 25, 2022

kibanamachine reopened this Mar 3, 2022

jbudz closed this as completed Mar 3, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Failing test: Chrome X-Pack UI Functional Tests.x-pack/test/functional/apps/uptime/synthetics_integration·ts - Uptime app with generated data When on the Synthetics Integration Policy Create Page displays custom UI "before all" hook for "should display policy view" #116522

Failing test: Chrome X-Pack UI Functional Tests.x-pack/test/functional/apps/uptime/synthetics_integration·ts - Uptime app with generated data When on the Synthetics Integration Policy Create Page displays custom UI "before all" hook for "should display policy view" #116522

kibanamachine commented Oct 27, 2021 •

edited

Loading

kibanamachine commented Oct 27, 2021

elasticmachine commented Oct 28, 2021

joshdover commented Oct 28, 2021

kibanamachine commented Oct 28, 2021

kibanamachine commented Oct 28, 2021

kibanamachine commented Oct 28, 2021

kibanamachine commented Oct 28, 2021

kibanamachine commented Oct 28, 2021

kibanamachine commented Oct 28, 2021

jbudz commented Oct 28, 2021

kibanamachine commented Oct 28, 2021

kibanamachine commented Oct 28, 2021

kibanamachine commented Oct 28, 2021

kibanamachine commented Oct 28, 2021

lucasfcosta commented Nov 18, 2021

dominiqueclarke commented Nov 23, 2021

lucasfcosta commented Dec 21, 2021

dominiqueclarke commented Jan 25, 2022

kibanamachine commented Feb 16, 2022

kibanamachine commented Feb 16, 2022

dominiqueclarke commented Feb 23, 2022

dominiqueclarke commented Feb 23, 2022

joshdover commented Feb 24, 2022

dominiqueclarke commented Feb 25, 2022

kibanamachine commented Mar 3, 2022

kibanamachine commented Mar 3, 2022

Failing test: Chrome X-Pack UI Functional Tests.x-pack/test/functional/apps/uptime/synthetics_integration·ts - Uptime app with generated data When on the Synthetics Integration Policy Create Page displays custom UI "before all" hook for "should display policy view" #116522

Failing test: Chrome X-Pack UI Functional Tests.x-pack/test/functional/apps/uptime/synthetics_integration·ts - Uptime app with generated data When on the Synthetics Integration Policy Create Page displays custom UI "before all" hook for "should display policy view" #116522

Comments

kibanamachine commented Oct 27, 2021 • edited Loading

kibanamachine commented Oct 27, 2021

elasticmachine commented Oct 28, 2021

joshdover commented Oct 28, 2021

kibanamachine commented Oct 28, 2021

kibanamachine commented Oct 28, 2021

kibanamachine commented Oct 28, 2021

kibanamachine commented Oct 28, 2021

kibanamachine commented Oct 28, 2021

kibanamachine commented Oct 28, 2021

jbudz commented Oct 28, 2021

kibanamachine commented Oct 28, 2021

kibanamachine commented Oct 28, 2021

kibanamachine commented Oct 28, 2021

kibanamachine commented Oct 28, 2021

lucasfcosta commented Nov 18, 2021

dominiqueclarke commented Nov 23, 2021

lucasfcosta commented Dec 21, 2021

dominiqueclarke commented Jan 25, 2022

kibanamachine commented Feb 16, 2022

kibanamachine commented Feb 16, 2022

dominiqueclarke commented Feb 23, 2022

dominiqueclarke commented Feb 23, 2022

joshdover commented Feb 24, 2022

dominiqueclarke commented Feb 25, 2022

kibanamachine commented Mar 3, 2022

kibanamachine commented Mar 3, 2022

kibanamachine commented Oct 27, 2021 •

edited

Loading