[Docs]Add Elastic Security flowchart to the docs

[benskelker]

Currently, there is no diagram illustrating the Elastic Security workflow. @MikePaquette and I started working on one for the 7.9 release (based on Mike's roadmap flow).

The draft flow is here: https://docs.google.com/drawings/d/1snRC0crIxcWdig3RsMkvKbIHq_lqZGpjymbpcT6O9rU/edit?usp=sharing

We'd like to get feedback on the flow from the other tech writers, and the product and UI teams. There's no need to worry about the diagram's aesthetics (unless you want to), as we'll send the approved flow to the design team.

Thanks,

cc @jmikell821, @Donnater, @marrasherrier, @dontcallmesherryli, @caitlinbetz, @jamiehynds, @shimonmodi

[benskelker]

The design team doesn't support this type of request (see https://github.com/elastic/Design/issues/2565), so we're on our own.

[dontcallmesherryli]

Lucidchart (part of the google suite) is a very handy application that makes pretty flowcharts and very easy to use. Not sure if you want to try using that @benskelker

[jmikell821]

@Donnater @benskelker please review the first draft Monina created.

[narcher7]

@Donnater @benskelker please review the first draft Monina created.

The information is good but I would like to see the shapes used more strategically. Specifically, I'm not sure the alerts, cases, and exceptions should be identified as "documents" when they're more similar to data shape in my mind. Otherwise, looks good.

[monina-n]

@Donnater @benskelker Thank you for the feedback. I agree that the shapes don't all match the expected conventions and think it would be advantageous to change them. I've included a list of the traditional conventions for each shape.

Some examples of changes could be:
Detection alerts, Cases, and Exceptions -> Data/Stored Data/Database(?)
Timeline investigations -> Process

I'm not sure which shape type is the best representation for each item so when there is a consensus, I can update and finalize the chart :)

[MikePaquette]

good points! We should indeed give some additional thought to the shapes we use.
Interestingly, events and alerts are actual documents in the Elasticsearch sense of the word.
The goal in the original diagram was to show that there were multiples of those things.

[benskelker]

Thanks @monina-n - I think it looks good. I don't know the conventions and I'm not sure about the meaning of the multiple document shape vs ES documents, so please feel free to change. I guess alerts, cases, and exceptions could be considered stored data. I thought the parallelogram represented input/output: https://en.wikipedia.org/wiki/Flowchart#Building_blocks

[monina-n]

quick question, what's the difference between "Timeline investigations" and "Timelines"?

[MikePaquette]

@monina-n the original idea was that the timeline investigation was a process that the analyst performed, drawing from (or adding to) a library of timeline objects (now includes timeline templates) to assist the process.

[monina-n]

@MikePaquette thank you for the clarification!

After talking with @dontcallmesherryli and looking up different flow chart shape conventions (1, 2), we realized that the current diagram doesn't fit any conventions since it's a mix of a user process diagram (ie. user chooses to investigate a timeline, create a case, etc.) and an architecture diagram (ie. endpoint sends data to the cluster automatically, events run through detection engine to determine if it's an alert).

There are two ways to approach the diagram. I'd like to ask what direction everyone prefers.

1. Keep the original diagram with an understanding it's not using standard shape conventions. Maybe include a key to explain meaning of shapes used.

2. Split up the diagram into two different diagrams for user process and architecture. This requires much more effort.

My personal opinion: I think the original diagram is still pretty clear and easy to follow as a user. I have some ideas about categorize the items and what shapes to use. Splitting it up into two diagram will take more time and effort, but it might be worth it so all future diagrams follow expected conventions. @dontcallmesherryli offered her insight how to split it up once she gets back from vacation on 7/27 if we go this route.

Please let me know your thoughts! I'd love to finish this up for everyone soon

@MikePaquette @benskelker @Donnater

[benskelker]

Thanks @monina-n - please don't feel rushed. I'd prefer this to be one diagram in the docs. @dontcallmesherryli asked me what we wanted the diagram for, maybe my answer helps:

It’s going to be part of the app overview section. Together with a short description under the diagram, it can provide a high-level overview of what the app can do and how different features work together, as well as be a place from which users can pivot to more specific stuff. I guess the audience is very general, anyone who uses or is interested in the app.

If you need to correct shapes to keep this as one diagram, that's fine.

[monina-n]

Hi all, here is a link to a Google Drive folder of the final draft of the security workflow chart. There's two versions- one with the original diagram language and one with more active language for the user processes. I've included screenshots here as well. Let me know if there's anything to be added or changed in the digram.

Original:

Active:

[benskelker]

Thanks very much - I think the Active version is great.

[MikePaquette]

@monina-n Whoa, this is looking great! The active language for user processes works really well, and the organization is really clean.

With this clean base, It really got me thinking on how to make this even better and more complete! I have a bunch of minor suggestions and a few additions in the attached mark-up.

The biggest adds are:

• to show the diversity of data sources we support on the left side of the diagram. These are available as modules and packages.
• adding a way to administer policies and configurations of hosts running Elastic Security at the top of the diagram
• Adding in the multiple analytics that support the detection engine
• Adding in our new-for-7.9 value lists into the diagram.

Please let me know if any of these changes don't make sense.
We really appreciate you taking the time to make this diagram accurate and helpful to our readers!

[monina-n]

happy monday! here's an updated version with the proposed changes. File can be found here

@MikePaquette

[MikePaquette]

Thanks @monina-n this looks great! Just a few final requests:

1. Change "Hosts running Elastic Security Endpoint" to "Hosts running Elastic Endpoint Security"
2. Change "Intrusion detection and preventions" to "Intrusion detection and prevention"
3. Change "via Logstash, Beats Modules, Elastic Packages" to "via Logstash, Beats modules, Ingest Manager integrations

Thanks!

[monina-n]

@MikePaquette done and done! file here

[benskelker]

Closing this. Thanks again @monina-n