[Maps][ML] Integration: part 1 POC extended

[alvarezmelissa87]

Summary

Extends and updates #90497
Incorporates latest maps changes (#112465, #112333) into initial draft PR (#111228)

Checklist

Delete any items that are not applicable to this PR.

• Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
• Documentation was added for features that require explanation or tutorials
• Unit or functional tests were updated or added to match the most common scenarios
• Any UI touched in this PR is usable by keyboard only (learn more about keyboard accessibility)
• Any UI touched in this PR does not create any new axe failures (run axe in browser: FF, Chrome)
• If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
• This renders correctly on smaller devices using a responsive layout. (You can test this in your browser)
• This was checked for cross-browser compatibility

Risk Matrix

Delete this section if it is not applicable to this PR.

Before closing this PR, invite QA, stakeholders, and other developers to identify risks that should be tested prior to the change/feature release.

When forming the risk matrix, consider some of the following examples and how they may potentially impact the change:

Risk	Probability	Severity	Mitigation/Notes
Multiple Spaces—unexpected behavior in non-default Kibana Space.	Low	High	Integration tests will verify that all features are still supported in non-default Kibana Space and when user switches between spaces.
Multiple nodes—Elasticsearch polling might have race conditions when multiple Kibana nodes are polling for the same tasks.	High	Low	Tasks are idempotent, so executing them multiple times will not result in logical error, but will degrade performance. To test for this case we add plenty of unit tests around this logic and document manual testing procedure.
Code should gracefully handle cases when feature X or plugin Y are disabled.	Medium	High	Unit tests will verify that any feature flag or plugin combination still results in our service operational.
See more potential risk examples

For maintainers

• This was checked for breaking API changes and was labeled appropriately

[alvarezmelissa87]

@thomasneirynck - looks like we need these exported here so ML can access them

[alvarezmelissa87]

@thomasneirynck - ran into some issues with the changes from #112465

I'm now always getting this failure (and a blank screen when I try to navigate to maps):

Looks like the failure is coming from the lazy load here https://github.com/elastic/kibana/blob/master/x-pack/plugins/maps/public/api/register.ts#L18 when registerSource is run.
The error is caught here https://github.com/elastic/kibana/blob/master/x-pack/plugins/maps/public/lazy_load_bundle/index.ts#L103

Looks like it's occurring when everything is being imported from kibana_services.ts (https://github.com/elastic/kibana/blob/master/x-pack/plugins/maps/public/kibana_services.ts#L19) because of the assumption that coreStart will always be there.

The contents in the file dependent on coreStart can't be accessed in the plugin setup right now.

It would work if we moved these functions back to (or expose them from both) the start function but it does seem they should be in the setup. Happy to discuss other potential solutions but as of now it won't work without some changes on the maps side.

cc @spalger as we discussed some of this.

[nreese]

@alvarezmelissa87 Thanks for identifying the issues with registerSource in setup. I have opened #114150 to fix the problem.

[kibanamachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request
• Commit: a9787f5
• Storybooks Preview

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
ml	1683	1697	+14

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
maps	201	207	+6

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
maps	2.7MB	2.7MB	+4.0B
ml	3.6MB	3.6MB	-21.8KB
total			-21.8KB

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id	before	after	diff
maps	29	27	-2

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
maps	41.7KB	41.9KB	+261.0B
ml	34.3KB	64.7KB	+30.5KB
total			+30.7KB

Unknown metric groups

API count

id	before	after	diff
maps	202	208	+6

async chunk count

id	before	after	diff
ml	25	24	-1

References to deprecated APIs

id	before	after	diff
ml	704	710	+6

History

• 💔 Build #157781 failed ed7b2d7

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[thomasneirynck]

This is really great. The main advantage of course is that it will allow ML-users to actually view their spatial-based ML jobs on a map. This will also help us internally. There is potential (e.g. "impossible travel"). Having the ability to easily map an ML-job will be invaluable to troubleshoot.

---

The register-source API should provide ordering

The ML-card now shows first. Which is cool :) . But inconsistent with the other soltion-cards, like the RUM or SIEM layers. imho Maps needs to introduce better ordering

^ ML card should be added with the other solution-cards here.

This is on the Maps-plugin end, not the ML-end.

---

The card should be disabled when the user does not have access to ML

The ML card is always enabled. It should not be when the functionality is not accessible . This check, should be at least the for checking enteprise-license. e.g.

kibana/x-pack/plugins/ml/public/maps/anomaly_layer_wizard.tsx

Lines 36 to 40 in a9787f5

	getIsDisabled: () => {
	// Do enterprise license check
	// Do check if user has access to job (namespace chec or whatever)
	return false;
	},

For an example, see

kibana/x-pack/plugins/maps/public/classes/sources/es_geo_line_source/layer_wizard.tsx

Lines 24 to 28 in a9ee83c

	disabledReason: REQUIRES_GOLD_LICENSE_MSG,
	icon: TracksLayerIcon,
	getIsDisabled: () => {
	return !getIsGoldPlus();
	},

This could also include other checks. (e.g. the Observability-layers check whether APM is configured)

Because the card is enabled, there is now a bug after clicking when the user does not hve an enterprise-license.

Uncaught (in promise) Error: Forbidden
    at Fetch.fetchResponse (:5601/dam/9007199254740991/bundles/core/core.entry.js:13463)
    at async interceptResponse (:5601/dam/9007199254740991/bundles/core/core.entry.js:13819)
    at async :5601/dam/9007199254740991/bundles/core/core.entry.js:13360

and an empty wizard.

---

Are tooltips useful?

The tooltips only include the record-score. I am wondering if there are other fields would be valuable.

e.g.

• Document id
• Fields used in the config (e.g. influencers, part of detectors arguments (?))
• partition-field values

---

The preconfigured color-ramps is not being picked up by the Maps-UX

^ this is a bug in Maps likely. Maps-team should provide guidance here on how to resolve.

---

Handle empty state in wizard

When there are no ML jobs available for selection in the wizard, maybe we should not show the combo-box, and instead show some exaplanation/hyperlink. E.g. a link to the ML anomaly explorer.

---

Enable time filtering

While I don't think full support for KQL is not necessary, it would be nice the ML-layer syncs with the time-picker.

---

Can we link out from the ML app to a pre-configured Maps app?

Anytime a job is "mappable", can we provide a hyperlink in the ML-app, something like "Explore in maps".

[kibana-ci]

💔 Build Failed

• Buildkite Build
• Commit: 31f7211
• Interpreting CI Failures
• Storybooks Preview

Failed CI Steps

• Linting (with types)

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
ml	1588	1604	+16

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
maps	206	212	+6

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
maps	2.6MB	2.6MB	+11.0B
ml	3.5MB	3.5MB	+18.4KB
total			+18.4KB

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id	before	after	diff
maps	30	28	-2

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
maps	38.5KB	38.7KB	+260.0B
ml	38.5KB	38.6KB	+122.0B
total			+382.0B

Unknown metric groups

API count

id	before	after	diff
maps	207	213	+6

async chunk count

id	before	after	diff
ml	25	27	+2

ESLint disabled line counts

id	before	after	diff
ml	94	97	+3

References to deprecated APIs

id	before	after	diff
ml	61	67	+6

Total ESLint disabled count

id	before	after	diff
ml	97	100	+3

History

• 💔 Build #14347 failed aee2671
• 💔 Build #13737 failed 38df4ba
• 💔 Build #13548 failed 47e9e34
• 💔 Build #13270 failed b1c8037
• 💔 Build #9796 failed 38a897e

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[alvarezmelissa87]

Closing in lieu of #122862