Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Fleet] Add showOnlyActiveDatastream to /packages/installed endpoint #192631

Merged
merged 4 commits into from
Sep 13, 2024
Merged

[Fleet] Add showOnlyActiveDatastream to /packages/installed endpoint #192631

merged 4 commits into from
Sep 13, 2024

Conversation

nchaulet
Copy link
Member

@nchaulet nchaulet commented Sep 11, 2024 •
edited by kibanamachine
Loading

Summary

Resolve #191990

Add a new option to installed package endpoint to only return active datastream.
the query params is a little different than in the original issue showOnlyActiveDataStreams as it seems more explict to me (feel free to suggest a better name)

When passing that new params the API will only return existing datastreams pattern.

Performance concerns

I tested this will all the package installed, and 200 datastreams and it seems there is no performance issue (tested locally)

Manual tests

You can install some package and verify that the datastreams are not in the installed pacakge response unless you add some data.

http GET http://localhost:5601/api/fleet/epm/packages/installed\?showOnlyActiveDataStreams\=true kbn-xsrf:xx elastic-api-version:2023-10-31   --auth=elastic:changeme

Todo

  • Add API integration tests
  • Add unit tests

@nchaulet nchaulet added release_note:skip Skip the PR/issue when compiling release notes Team:Fleet Team label for Observability Data Collection Fleet team labels Sep 11, 2024
@nchaulet nchaulet self-assigned this Sep 11, 2024
@obltmachine
Copy link

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

  • /oblt-deploy : Deploy a Kibana instance using the Observability test environments.
  • run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

@nchaulet nchaulet force-pushed the feature-showOnlyActiveDatastream-installed-package branch from f3125b6 to 9c24e79 Compare September 11, 2024 19:24
@nchaulet nchaulet marked this pull request as ready for review September 12, 2024 13:37
@nchaulet nchaulet requested a review from a team as a code owner September 12, 2024 13:37
@elasticmachine
Copy link
Contributor

Pinging @elastic/fleet (Team:Fleet)

@kibana-ci
Copy link
Collaborator

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

  • [job] [logs] FTR Configs #3 / task_manager with mget task claimer task partitions should run tasks with partitions assigned to this kibana node

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @nchaulet

@jen-huang
Copy link
Contributor

#191990 seems to be asking for additional data (active or not) to be added to each data stream that is returned by this API when a query param is set, rather than a query param that filters the data streams returned. Does the approach in this PR still work for the ES|QL editor use case?

@nchaulet
Copy link
Member Author

nchaulet commented Sep 12, 2024 •
edited
Loading

#191990 seems to be asking for additional data (active or not) to be added to each data stream that is returned by this API when a query param is set, rather than a query param that filters the data streams returned. Does the approach in this PR still work for the ES|QL editor use case?

Yes that a good point, @drewdaemon @stratoula does this (filtering non active) work for you or adding a new isActive property is better suited for your use case?

@stratoula
Copy link
Contributor

stratoula commented Sep 13, 2024 •
edited
Loading

@nchaulet filtering them out is even better I think so we are totally fine with this approach 🙌

@stratoula stratoula mentioned this pull request Sep 13, 2024
Merged
1 task
jen-huang
jen-huang approved these changes Sep 13, 2024
View reviewed changes
Copy link
Contributor

@jen-huang jen-huang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code LGTM

@nchaulet nchaulet merged commit fa74847 into elastic:main Sep 13, 2024
27 checks passed
@nchaulet nchaulet deleted the feature-showOnlyActiveDatastream-installed-package branch September 13, 2024 16:36
@kibanamachine kibanamachine added v9.0.0 backport:skip This commit does not require backporting labels Sep 13, 2024
@drewdaemon
Copy link
Contributor

Thank you thank you! This is a huge help to us 🙏

@nchaulet nchaulet added v8.16.0 backport:version Backport to applied version labels and removed backport:skip This commit does not require backporting labels Sep 13, 2024
@kibanamachine
Copy link
Contributor

💔 All backports failed

Status Branch Result
❌ 8.16 The branch "8.16" does not exist

Manual backport

To create the backport manually run:

node scripts/backport --pr 192631

Questions ?

Please refer to the Backport tool documentation

@nchaulet nchaulet added backport:prev-major Backport to (8.x, 8.15) the previous major branch and all later branches still in development and removed v8.16.0 backport:version Backport to applied version labels labels Sep 13, 2024
nchaulet added a commit to nchaulet/kibana that referenced this pull request Sep 13, 2024
@nchaulet
[Fleet] Add showOnlyActiveDatastream to /packages/installed endpoint (e…
43ea968 
…lastic#192631)

(cherry picked from commit fa74847)
@nchaulet nchaulet mentioned this pull request Sep 13, 2024
Closed
@nchaulet
Copy link
Member Author

💚 All backports created successfully

Status Branch Result
✅ 8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Sep 13, 2024
@nchaulet
[Fleet] Add showOnlyActiveDatastream to /packages/installed endpoint (e…
60f0130 
…lastic#192631)

(cherry picked from commit fa74847)
@kibanamachine kibanamachine mentioned this pull request Sep 13, 2024
Closed
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Sep 13, 2024
@nchaulet
[Fleet] Add showOnlyActiveDatastream to /packages/installed endpoint (e…
04f83f7 
…lastic#192631)

(cherry picked from commit fa74847)
@kibanamachine kibanamachine mentioned this pull request Sep 13, 2024
Merged
@kibanamachine
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
✅ 8.15
✅ 8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

@nchaulet nchaulet added backport:skip This commit does not require backporting and removed backport:prev-major Backport to (8.x, 8.15) the previous major branch and all later branches still in development labels Sep 13, 2024
kibanamachine added a commit that referenced this pull request Sep 13, 2024
@kibanamachine @nchaulet
[8.x] [Fleet] Add showOnlyActiveDatastream to /packages/installed end…
a45be49 
…point (#192631) (#192927)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[Fleet] Add showOnlyActiveDatastream to /packages/installed endpoint
(#192631)](#192631)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Nicolas
Chaulet","email":"nicolas.chaulet@elastic.co"},"sourceCommit":{"committedDate":"2024-09-13T16:36:41Z","message":"[Fleet]
Add showOnlyActiveDatastream to /packages/installed endpoint
(#192631)","sha":"fa748474c42ed0111dde735fb2d51d202ed86a9e","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Fleet","v9.0.0","backport:prev-major"],"title":"[Fleet]
Add showOnlyActiveDatastream to /packages/installed
endpoint","number":192631,"url":"https://github.com/elastic/kibana/pull/192631","mergeCommit":{"message":"[Fleet]
Add showOnlyActiveDatastream to /packages/installed endpoint
(#192631)","sha":"fa748474c42ed0111dde735fb2d51d202ed86a9e"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/192631","number":192631,"mergeCommit":{"message":"[Fleet]
Add showOnlyActiveDatastream to /packages/installed endpoint
(#192631)","sha":"fa748474c42ed0111dde735fb2d51d202ed86a9e"}}]}]
BACKPORT-->

Co-authored-by: Nicolas Chaulet <nicolas.chaulet@elastic.co>
@drewdaemon drewdaemon mentioned this pull request Sep 16, 2024
Merged
markov00 pushed a commit to markov00/kibana that referenced this pull request Sep 18, 2024
@nchaulet @markov00
[Fleet] Add showOnlyActiveDatastream to /packages/installed endpoint (e…
8fbfa2f 
…lastic#192631)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jen-huang jen-huang jen-huang approved these changes

Assignees

@nchaulet nchaulet

Labels
backport:skip This commit does not require backporting release_note:skip Skip the PR/issue when compiling release notes Team:Fleet Team label for Observability Data Collection Fleet team v8.16.0 v9.0.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Fleet] Support isActive property for each dataStreams entry returned by /api/fleet/epm/packages/installed
8 participants
@nchaulet @obltmachine @elasticmachine @kibana-ci @jen-huang @stratoula @drewdaemon @kibanamachine