Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ML] Cardinality validation API integration tests #65971

Merged
merged 3 commits into from
May 11, 2020
Merged

[ML] Cardinality validation API integration tests #65971

merged 3 commits into from
May 11, 2020

Conversation

darnautov
Copy link
Contributor

@darnautov darnautov commented May 11, 2020 •
edited
Loading

Summary

Part of #63700.

  • Adds API integration tests for cardinality validation of anomaly detection jobs.

  • TS refactoring

  • Unit or functional tests were updated or added to match the most common scenarios

@darnautov darnautov added :ml Feature:Anomaly Detection ML anomaly detection v8.0.0 release_note:skip Skip the PR/issue when compiling release notes v7.8.0 labels May 11, 2020
@darnautov darnautov requested a review from a team as a code owner May 11, 2020 08:28
@darnautov darnautov self-assigned this May 11, 2020
@elasticmachine
Copy link
Contributor

Pinging @elastic/ml-ui (:ml)

@peteharverson peteharverson mentioned this pull request May 11, 2020
Closed
90 tasks
walterra
walterra approved these changes May 11, 2020
View reviewed changes
Copy link
Contributor

@walterra walterra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, also did a local test. Only remark is about the non-null assertions, can we try to avoid them? (https://github.com/elastic/kibana/blob/master/STYLEGUIDE.md#avoid-non-null-assertions).

@darnautov
Copy link
Contributor Author

LGTM, also did a local test. Only remark is about the non-null assertions, can we try to avoid them? (https://github.com/elastic/kibana/blob/master/STYLEGUIDE.md#avoid-non-null-assertions).

Yes, I tried to avoid it but didn't find a solution. The problem is validateJobObject function takes job as input and throws an error in case it's null. Hence in the code after this function call, we can be sure it's defined. However, TS is not able to recognize it and cast the type. It's possible to change this function to be a TS guard, but this it's used in other API endpoint so I decided not to make this change.

jgowdyelastic
jgowdyelastic approved these changes May 11, 2020
View reviewed changes
Copy link
Member

@jgowdyelastic jgowdyelastic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a couple of ts nitpicks, ignore if you want, otherwise LGTM.
It's nice to see further removal of lodash!

x-pack/plugins/ml/server/models/job_validation/validate_cardinality.ts Outdated
function isScriptField(job, fieldName) {
const scriptFields = Object.keys(_.get(job, 'datafeed_config.script_fields', {}));
function isScriptField(job: CombinedJob, fieldName: string): boolean {
const scriptFields = Object.keys(job?.datafeed_config?.script_fields ?? {});
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit, it doesn't look like job will be undefined here, datafeed_config also shouldn't be undefined.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 90ba7a4

x-pack/plugins/ml/server/models/job_validation/validate_cardinality.ts Outdated
const uniqueFieldNames = _.uniq(relevantDetectors.map(f => f[fieldName]));
const uniqueFieldNames: string[] = [
...new Set(relevantDetectors.map(f => f[fieldName])),
] as string[];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit, could be const uniqueFieldNames = [...new Set(relevantDetectors.map(f => f[fieldName] as string))];

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 90ba7a4

@walterra
Copy link
Contributor

@darnautov to make validateJobObject() a type guard sounds good, can of course be done another time! 👍

@darnautov
Copy link
Contributor Author

@darnautov to make validateJobObject() a type guard sounds good, can of course be done another time! 👍

@walterra validateJobObject is a TS guard now, check 90ba7a4

peteharverson
peteharverson approved these changes May 11, 2020
View reviewed changes
Copy link
Contributor

@peteharverson peteharverson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

pheyos
pheyos approved these changes May 11, 2020
View reviewed changes
Copy link
Member

@pheyos pheyos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kibanamachine
Copy link
Contributor

💚 Build Succeeded

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@darnautov darnautov merged commit d1e4d37 into elastic:master May 11, 2020
@darnautov darnautov deleted the ML-63700-validation-tests branch May 11, 2020 13:57
darnautov added a commit to darnautov/kibana that referenced this pull request May 11, 2020
@darnautov
[ML] Cardinality validation API integration tests (elastic#65971)
ee9a1cb 
* [ML] refactor validate_cardinality to TS

* [ML] cardinality api integration tests

* [ML] resolve PR comments, validateJobObject as TS guard
@darnautov darnautov mentioned this pull request May 11, 2020
Merged
darnautov added a commit to darnautov/kibana that referenced this pull request May 11, 2020
@darnautov
[ML] Cardinality validation API integration tests (elastic#65971)
c204104 
* [ML] refactor validate_cardinality to TS

* [ML] cardinality api integration tests

* [ML] resolve PR comments, validateJobObject as TS guard
@darnautov darnautov mentioned this pull request May 11, 2020
Merged
darnautov added a commit that referenced this pull request May 12, 2020
@darnautov
[ML] Cardinality validation API integration tests (#65971) (#66047)
7d00e48 
* [ML] refactor validate_cardinality to TS

* [ML] cardinality api integration tests

* [ML] resolve PR comments, validateJobObject as TS guard
darnautov added a commit that referenced this pull request May 12, 2020
@darnautov
[ML] Cardinality validation API integration tests (#65971) (#66048)
ba9db32 
* [ML] refactor validate_cardinality to TS

* [ML] cardinality api integration tests

* [ML] resolve PR comments, validateJobObject as TS guard
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@walterra walterra walterra approved these changes

@pheyos pheyos pheyos approved these changes

@peteharverson peteharverson peteharverson approved these changes

@jgowdyelastic jgowdyelastic jgowdyelastic approved these changes

Assignees

@darnautov darnautov

Labels
Feature:Anomaly Detection ML anomaly detection :ml release_note:skip Skip the PR/issue when compiling release notes test-api-integration v7.8.0 v8.0.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@darnautov @elasticmachine @walterra @kibanamachine @pheyos @peteharverson @jgowdyelastic