[Fleet] Create Synthetics migration for 8.8.0 #154952

dominiqueclarke · 2023-04-13T20:01:02Z

Summary

Resolves #155215
Resolves #142653

Handles two primary migrations for Synthetics integration policies.

Rounds deprecated schedules to supported schedules
Transforms old deprecated throttling schema 5u/3u/20l to support JSON schema { download: 5, upload: 3, latency: 20 }

Schedule migration

Before 16m schedule

After

Before 4m schedule

After

Before 8m schedule

After

Before 2m schedule

After

Throttling migration

Before throttling: false

After

Before custom throttling

After

Before default throttling

After

Testing

Check out the 8.7.0 branch
Create a Synthetics private location at app/synthetics/settings/private-locations.
Create a monitor, configured at that private location, with invalid schedules, ideally 1 or each type (http, icmp, browser, tcp) with an invalid schedule (for example, 2, 8, 11, 16, 333, etc) at app/uptime/add-monitor. Note: you must use Uptime to create monitors with arbitrary schedules. The Synthetics app will not let you.
Create a browser monitor, configured with your private location, with throttling turned off.
Create a browser monitor, configured for your private location, with a custom throttling profile.
Check out this PR and wait for saved object migration to run
Navigate to the agent policy for your monitor. Confirm the schedules were updated to supported schedules. Confirm the throttling configs now appear in yaml. Confirm that throttling: false remains for the throttling config that was turned off.

elasticmachine · 2023-04-13T20:01:09Z

Pinging @elastic/fleet (Team:Fleet)

apmmachine · 2023-04-13T20:01:19Z

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

/oblt-deploy : Deploy a Kibana instance using the Observability test environments.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

dominiqueclarke · 2023-04-13T20:01:56Z

Sorry for pinging folks. Meant to put this in draft.

…thub.com/dominiqueclarke/kibana into feat/synthetics-package-policy-migration

elasticmachine · 2023-04-20T01:43:09Z

Pinging @elastic/uptime (Team:uptime)

shahzad31 · 2023-04-20T12:24:02Z

x-pack/plugins/fleet/server/saved_objects/migrations/synthetics/to_v8_8_0.ts

+  throttling: string
+): { download: number; upload: number; latency: number } => {
+  try {
+    const [download = 5, upload = 3, latency = 20] = throttling.match(/\d+\.?\d*/g) || [];


TIL . Looks like fancy string split :)

shahzad31 · 2023-04-20T12:26:09Z

x-pack/plugins/synthetics/common/formatters/browser/formatters.ts

-  if (!throttling || throttling?.id === 'no-throttling' || !throttling?.value) {
-    return 'false';
-  }


i think we need to keep this when throttling is disabled.

YOu are absolutely right. Thanks for the catch. I think I got rid of this in a bad merge conflict resolution. I'll write a unit test to make sure this doesn't happen again.

shahzad31

LGTM !!

Tested schedule and throttling migration !!

x-pack/plugins/fleet/server/types/models/package_policy.ts

kibana-ci · 2023-04-20T16:39:20Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: a2afcb9

Failed CI Steps

FTR Configs #55

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`synthetics`	1.2MB	1.2MB	+38.0B

Unknown metric groups

ESLint disabled line counts

id	before	after	diff
`securitySolution`	394	397	+3

Total ESLint disabled count

id	before	after	diff
`securitySolution`	474	477	+3

History

💔 Build #121620 failed d95376a
💛 Build #121581 was flaky 4b6496d
💛 Build #121378 was flaky d9c33dc
💔 Build #121364 failed f3cb08a
💔 Build #121328 failed 2b8ccf2

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

kpollich

Fleet changes 🚀

## Summary Resolves elastic#155215 Resolves elastic#142653 Handles two primary migrations for Synthetics integration policies. 1. Rounds deprecated schedules to supported schedules 2. Transforms old deprecated throttling schema `5u/3u/20l` to support JSON schema `{ download: 5, upload: 3, latency: 20 }` Schedule migration --- Before 16m schedule <img width="200" alt="Screen Shot 2023-04-19 at 6 17 35 PM" src="https://user-images.githubusercontent.com/11356435/233211993-086ce51e-a056-407f-900b-15638238a243.png"> After <img width="214" alt="Screen Shot 2023-04-19 at 6 44 08 PM" src="https://user-images.githubusercontent.com/11356435/233215688-e049e2b7-6958-4376-9362-a192aa8e94db.png"> Before 4m schedule <img width="190" alt="Screen Shot 2023-04-19 at 6 17 29 PM" src="https://user-images.githubusercontent.com/11356435/233211995-6b7b2927-6851-4947-ab8c-7fb041da03e1.png"> After <img width="208" alt="Screen Shot 2023-04-19 at 6 44 30 PM" src="https://user-images.githubusercontent.com/11356435/233215685-89f093e6-3308-4103-9b98-5acb7a9b5a0a.png"> Before 8m schedule <img width="202" alt="Screen Shot 2023-04-19 at 6 17 23 PM" src="https://user-images.githubusercontent.com/11356435/233211997-9a60c54c-4867-4ccd-a8eb-5b62d59060bf.png"> After <img width="201" alt="Screen Shot 2023-04-19 at 6 44 22 PM" src="https://user-images.githubusercontent.com/11356435/233215687-f3a900cf-b3ec-44f5-b84b-d292dbad623d.png"> Before 2m schedule <img width="193" alt="Screen Shot 2023-04-19 at 6 17 16 PM" src="https://user-images.githubusercontent.com/11356435/233211999-3d42ad71-b72b-4876-911e-5d79564f2351.png"> After <img width="194" alt="Screen Shot 2023-04-19 at 6 43 55 PM" src="https://user-images.githubusercontent.com/11356435/233215690-fd3f13dc-5e32-4904-b804-cbcfcabf0760.png"> Throttling migration --- Before throttling: false <img width="163" alt="Screen Shot 2023-04-19 at 6 17 00 PM" src="https://user-images.githubusercontent.com/11356435/233212002-3a891b25-fc2e-4cce-a730-abf8695d5423.png"> After <img width="185" alt="Screen Shot 2023-04-19 at 6 49 50 PM" src="https://user-images.githubusercontent.com/11356435/233216370-eed97645-26e8-44f2-8f72-8d1e19f39c35.png"> Before custom throttling <img width="274" alt="Screen Shot 2023-04-19 at 6 16 54 PM" src="https://user-images.githubusercontent.com/11356435/233212004-a9fe82fc-d23a-4d54-a5ad-20971c3df211.png"> After <img width="169" alt="Screen Shot 2023-04-19 at 6 49 44 PM" src="https://user-images.githubusercontent.com/11356435/233216389-f2803ca9-3429-4f85-b1ee-0f5a8e02db92.png"> Before default throttling <img width="212" alt="Screen Shot 2023-04-19 at 6 16 48 PM" src="https://user-images.githubusercontent.com/11356435/233212007-d9fbcae7-6f6e-4cd0-b909-629cfc72b7dc.png"> After <img width="181" alt="Screen Shot 2023-04-19 at 6 49 35 PM" src="https://user-images.githubusercontent.com/11356435/233216412-70336d0e-1e45-4809-bd92-c4524a144b99.png"> ### Testing 1. Check out the 8.7.0 branch 2. Create a Synthetics private location at `app/synthetics/settings/private-locations`. 3. Create a monitor, configured at that private location, with invalid schedules, ideally 1 or each type (http, icmp, browser, tcp) with an invalid schedule (for example, 2, 8, 11, 16, 333, etc) at `app/uptime/add-monitor`. Note: you must use Uptime to create monitors with arbitrary schedules. The Synthetics app will not let you. 4. Create a browser monitor, configured with your private location, with throttling turned off. 5. Create a browser monitor, configured for your private location, with a custom throttling profile. 6. Check out this PR and wait for saved object migration to run 7. Navigate to the agent policy for your monitor. Confirm the schedules were updated to supported schedules. Confirm the throttling configs now appear in yaml. Confirm that `throttling: false` remains for the throttling config that was turned off.

fleet - create generic migration for 8.8.0

e2efc95

dominiqueclarke requested a review from a team as a code owner April 13, 2023 20:01

botelastic bot added the Team:Fleet Team label for Observability Data Collection Fleet team label Apr 13, 2023

dominiqueclarke marked this pull request as draft April 13, 2023 20:01

dominiqueclarke added 12 commits April 13, 2023 16:03

update synthetics migration

7f18b81

add schedule migration

92af081

add throttling migration

929cdb7

add throttling migration

fee725d

adjust throttling format

abe1b08

Merge branch 'main' into feat/synthetics-package-policy-migration

2b8ccf2

adjust migration

a9c100c

adjust unit test

1d93355

merge upstream

c979823

adjust tests

1e90adf

Merge branch 'feat/synthetics-package-policy-migration' of https://gi…

f3cb08a

…thub.com/dominiqueclarke/kibana into feat/synthetics-package-policy-migration

adjust types

d9c33dc

dominiqueclarke added Team:Uptime - DEPRECATED Synthetics & RUM sub-team of Application Observability v8.8.0 release_note:deprecation enhancement New value added to drive a business result labels Apr 20, 2023

dominiqueclarke changed the title ~~fleet - create generic migration for 8.8.0~~ [Fleet] Create Synthetics migration for 8.8.0 Apr 20, 2023

dominiqueclarke marked this pull request as ready for review April 20, 2023 01:43

dominiqueclarke requested a review from a team as a code owner April 20, 2023 01:43

shahzad31 reviewed Apr 20, 2023

View reviewed changes

adjust browser formatters to support throttling: false

4b6496d

shahzad31 approved these changes Apr 20, 2023

View reviewed changes

update PackagePolicyBaseSchema

d95376a

dominiqueclarke commented Apr 20, 2023

View reviewed changes

x-pack/plugins/fleet/server/types/models/package_policy.ts Outdated Show resolved Hide resolved

Update x-pack/plugins/fleet/server/types/models/package_policy.ts

a2afcb9

kpollich approved these changes Apr 20, 2023

View reviewed changes

dominiqueclarke merged commit b494716 into elastic:main Apr 21, 2023

kibanamachine added the backport:skip This commit does not require backporting label Apr 21, 2023

dominiqueclarke deleted the feat/synthetics-package-policy-migration branch April 21, 2023 13:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Fleet] Create Synthetics migration for 8.8.0 #154952

[Fleet] Create Synthetics migration for 8.8.0 #154952

dominiqueclarke commented Apr 13, 2023 •

edited

Loading

elasticmachine commented Apr 13, 2023

apmmachine commented Apr 13, 2023

dominiqueclarke commented Apr 13, 2023

elasticmachine commented Apr 20, 2023

shahzad31 Apr 20, 2023

shahzad31 Apr 20, 2023

dominiqueclarke Apr 20, 2023

shahzad31 left a comment

kibana-ci commented Apr 20, 2023

ESLint disabled line counts

Total ESLint disabled count

kpollich left a comment

[Fleet] Create Synthetics migration for 8.8.0 #154952

[Fleet] Create Synthetics migration for 8.8.0 #154952

Conversation

dominiqueclarke commented Apr 13, 2023 • edited Loading

Summary

Schedule migration

Throttling migration

Testing

elasticmachine commented Apr 13, 2023

apmmachine commented Apr 13, 2023

🤖 GitHub comments

dominiqueclarke commented Apr 13, 2023

elasticmachine commented Apr 20, 2023

shahzad31 Apr 20, 2023

Choose a reason for hiding this comment

shahzad31 Apr 20, 2023

Choose a reason for hiding this comment

dominiqueclarke Apr 20, 2023

Choose a reason for hiding this comment

shahzad31 left a comment

Choose a reason for hiding this comment

kibana-ci commented Apr 20, 2023

💛 Build succeeded, but was flaky

Failed CI Steps

Metrics [docs]

Async chunks

ESLint disabled line counts

Total ESLint disabled count

History

kpollich left a comment

Choose a reason for hiding this comment

dominiqueclarke commented Apr 13, 2023 •

edited

Loading