Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ML][Fleet] Update Transform installation mechanism to support upgrade paths #142920

Merged
merged 25 commits into from
Dec 21, 2022
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
4a0ab13
[ML] Fix template being overridden
qn895 Oct 6, 2022
93c87ac
Clean up
qn895 Nov 9, 2022
61f033e
Clean up
qn895 Nov 2, 2022
f450566
Modify alias logic
qn895 Nov 2, 2022
adcbd4c
Add logic for adding aliases, order of installing transforms, updatin…
qn895 Nov 8, 2022
8ca3a2e
Wrap up for testing
qn895 Nov 9, 2022
af4db32
Merge upstream/main into branch
qn895 Nov 9, 2022
36646d1
Remove @todos since they are addressed
qn895 Nov 9, 2022
8e2a6ff
[TO REVERT] Ignore 400 for ingest pipeline deletion
qn895 Nov 9, 2022
49d1f59
Merge remote-tracking branch 'upstream/main' into ml-fleet-fix-templa…
qn895 Nov 14, 2022
cf1e93b
Remove console
qn895 Nov 14, 2022
ab3444f
[TODO] Update tests
qn895 Nov 15, 2022
0abab6d
Merge remote-tracking branch 'upstream/main' into ml-fleet-fix-templa…
qn895 Dec 5, 2022
ecbb8c1
Fix types, tests
qn895 Dec 5, 2022
6984e15
Remove index as a fleet setting, add more tests
qn895 Dec 5, 2022
8fb4a73
Merge branch 'main' into ml-fleet-fix-templates-being-overriden
kibanamachine Dec 6, 2022
d758b5b
Ensures it supports version downgrades & tests for downgrades
qn895 Dec 6, 2022
d6a22d2
Merge remote-tracking branch 'upstream/main' into ml-fleet-fix-templa…
qn895 Dec 6, 2022
6ee4cce
Update epm-package snapshot for saved_objects migrations
qn895 Dec 7, 2022
306a2e1
Merge remote-tracking branch 'upstream/main' into ml-fleet-fix-templa…
qn895 Dec 7, 2022
f09f995
Merge upstream into branch
qn895 Dec 14, 2022
e1a0050
Change to debug.error
qn895 Dec 14, 2022
e56481c
Update to delete old transform, and old dest index if upgrading from …
qn895 Dec 20, 2022
76402ac
Merge remote-tracking branch 'upstream/main' into ml-fleet-fix-templa…
qn895 Dec 20, 2022
a99e62c
Merge branch 'main' into ml-fleet-fix-templates-being-overriden
kibanamachine Dec 21, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions x-pack/plugins/fleet/server/saved_objects/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -254,6 +254,7 @@ const getSavedObjectTypes = (
properties: {
id: { type: 'keyword' },
type: { type: 'keyword' },
version: { type: 'keyword' },
},
},
installed_kibana: {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -46,11 +46,12 @@ export async function deletePipeline(esClient: ElasticsearchClient, id: string):
// '*' shouldn't ever appear here, but it still would delete all ingest pipelines
if (id && id !== '*') {
try {
await esClient.ingest.deletePipeline({ id });
// @TODO: Remove ignore 400
await esClient.ingest.deletePipeline({ id }, { ignore: [400] });
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the way to remove the 400 is to set the index.default_pipeline setting on the old destination index to the empty string before deleting the pipeline. See elastic/elasticsearch#32286 (comment).

} catch (err) {
// Only throw if error is not a 404 error. Sometimes the pipeline is already deleted, but we have
// duplicate references to them, see https://github.com/elastic/kibana/issues/91192
if (err.statusCode !== 404) {
if (err.statusCode !== 404 || err.statusCode !== 400) {
throw new FleetError(`error deleting pipeline ${id}: ${err}`);
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,8 @@
*/

export { getAsset } from '../../archive';

// Index alias that points to just one destination index from the latest package version
export const TRANSFORM_DEST_IDX_ALIAS_LATEST_SFX = '.latest';
// Index alias that points to all of the destination indices from all the package versions
export const TRANSFORM_DEST_IDX_ALIAS_ALL_SFX = '.all';
Loading