Add ML jobs installed via Elastic integrations to docs

[susan-shu-c]

Description

Currently, the prebuilt jobs reference page under Anomaly detection with machine learning only covers standalone jobs.

However, anomaly detection jobs can also be installed via Elastic integrations.

To make it easier for users to view these jobs all in one place, we'd like to add the jobs from integrations to one place.

Primarily, the change will be to these pages.

• https://www.elastic.co/guide/en/security/8.9/machine-learning.html
• https://www.elastic.co/guide/en/security/8.9/prebuilt-ml-jobs.html

Changes (proposed)

Anomaly Detection with ML

• Add sentence at beginning,
• Add section below “pre-built rules” to explain what advanced analytics use case packs are, introducing integration packages that install jobs – similar to Apoorva/Kseniia/Steve said

Prebuilt job reference → rename

• Add section and lists of integration packages, e.g. Security: Integration packages/ advanced analytics – similar to what Kirti/Apoorva said
• For packages, link to docs page
• Linking to code, is it useful? In the Job
• Use the columns “prebuilt rule reference” for and related integrations col. Remove datafeeds col.
  • Info is in: https://github.com/elastic/kibana/blob/main/x-pack/plugins/ml/server/models/data_recognizer/modules/security_packetbeat/ml/packetbeat_rare_dns_question.json
• Mention the related integration in the intro paragraph in each subsection

Update, seems that part of the code edits are here:

• https://github.com/elastic/stack-docs/blob/main/docs/en/stack/ml/anomaly-detection/ootb-ml-jobs-siem.asciidoc
• And I need to check if by modifying the stack-docs pages, will it break the automations in security-docs rules page (since the rules page reads from the jobs with scripts)

Acceptance Test Criteria

List all the ATC of each action and its intended result.
For example: As a user, when [action (e.g., viewing, clicking, selecting, etc.)] I should [insert the expected result].

Notes

• Add all appropriate labels to the issue, especially the version number label.
• Be sure to add any necessary screenshots, code text or console commands for clarity.
• Include any conditions or caveats that may affect customers.

Related ticket

• Expand ML Job/Rule documentation to include Related Integrations and Setup information #2974

[susan-shu-c]

@terrancedejesus, I saw that you made the most recent changes to the prebuilt jobs reference pages, and that the rules are automated off jobs' info via these scripts: https://github.com/elastic/security-docs/tree/main/prebuilt-rules-scripts

I've tried installing the required packages, but wanted to check in to see how we can introduce adding the jobs from Elastic integrations into this doc page without breaking the scripts?

Since the scripts read from very specific locations, I want to see if we can add to them, or perhaps just add this manually (hopefully not though!)

The idea is to add more sections like this: (from https://docs.elastic.co/integrations/problemchild#anomaly-detection-jobs)

[susan-shu-c]

Quick update:

I've made PRs but still need to confirm if there is more information to add, will follow up this week.

[susan-shu-c]

PRs

• Add package ml jobs stack-docs#2496
• Update ML page with anomaly detection jobs from Elastic integrations #3648

[susan-shu-c]

Update

Edit docs in security-docs and stack-docs and gotten reviews
Discussing with documentation/tech writer team for final approval

[susan-shu-c]

Closing as these PRs have been merged.

PRs

• Add package ml jobs stack-docs#2496
• Update ML page with anomaly detection jobs from Elastic integrations #3648