EUIfying Kibana

[cjcenizal]

We're going to rebuild Kibana's UI using EUI Framework components, achieving two goals:

• An updated visual style. We want to update Kibana's look-and-feel to match the "K7" designs coming from the @elastic/kibana-design team.
• Greatly reduced technical debt. We want to migrate Angular code to React (resulting in easier-to-test UI code and a clearer data flow throughout our codebase), and remove our dependency upon Bootstrap and custom LESS.

We have two requirements for the process of getting there:

• Stability. This process can't introduce any visual regressions or inconsistencies into our UI.
• Incremental progress. We should be able to make granular changes without impeding feature development or bug fixes.

The game plan

The K7 look-and-feel is a huge step forward but clashes with our existing look-and-feel. Fortunately, @snide designed the Sass portion of the EUI codebase to be themeable. We'll build a "K6 theme" in EUI (which is essentially a collection of Sass variables), which will apply the same look-and-feel we currently have in Kibana to the EUI components. This will let us add EUI as an NPM dependency to Kibana and we'll have the ability to use EUI React components throughout Kibana and X-Pack without any visual inconsistency.

Once we've converted our entire UI layer to consist solely of EUI components, we'll be able to remove the K6 theme variables and the entire Kibana UI will bear the K7 visual style. Work complete!

See #14948 for the original proof-of-concept of this idea and resulting discussion.

Who will be responsible?

Ultimately, the entire team will need to participate in this undertaking because of its enormous scope. Initially, I'll lead a small working group of engineers (@chrisronline, @nreese, @bmcconaghy) which will take point on this process and be responsible for identifying patterns, best practices, and other guidelines to share with the team at large.

We'll begin by executing this process against a few choice parts of the codebase. We'll choose areas of the codebase which have a limited scope, can be converted using existing EUI components, and are currently built in Angular. The goal of this exercise will be to suss out the best practices and technical challenges that come with this conversion. We're tracking progress at #15721.

We'll document what we learn throughout this exercise and fold in the insights gleaned by the @elastic/kibana-management team during their Reactification spike. Once we feel confident we understand the conversion process and its gotchas, we'll expand the process to the entire team. We'll work with each functional area team to define roadmap items and track progress on the release schedule. As we move forward we'll provide support and guidance to the greater team while picking up cross-cutting roadmap items (e.g. Notifications, Timepicker). As engineers discover a need for components, they'll collaborate with @elastic/kibana-design and be responsible for building these components for general use within EUI.

The nitty-gritty

Immediate tasks

• Audit and roadmap. We need to perform a full audit of the codebase and put together a roadmap with as much granularity as possible. ([META] EUIfication #15721)
• Convert KUI components to EUI. We'll go through the codebase and migrate all UIs built with KUI to use EUI. This will allow us to remove the KUI codebase, establish some EUI patterns immediately, and use the conversion process to provide guidelines for third-party plugin developers who may need to perform similar conversions.
• CSS reset. We need to replace the existing CSS reset in Kibana with the EUI reset and fix all visual regressions (EUI CSS reset regression audit #15315).
• Create version branches and add as dependencies to Kibana. We need to merge the K6 theme into master (DO NOT MERGE: Add K6 overrides to light theme.  eui#153) and create a 6.x branch. Then we need to add these branches as dependencies to the corresponding branches in Kibana.
  • We need to update EUI to support the K6 theme in the documentation site.
• Dark theme parity. We need to figure out a system that supports the dark theme in Dashboard. A couple options are:
  • 🥇 Just load the dark theme CSS when the user opens a dark theme Dashboard. (EUI: Reset CSS #15486)
  • Update EUI to build a CSS file which consists of the dark theme selectors wrapped in a .theme-dark class.
• Enable Sass to be used in Kibana. This will allow us to import the EUI Sass instead of using the precompiled CSS. This will give us access to variables and mixins for Kibana-specific components and overrides (Add support for SCSS compilation during development, but ship as compiled CSS #15274).
• EuiTooltip. This is a blocker. We need a EUI replacement for our current Angular tooltips. (tooltip component eui#174). After discussing this with @nreese we decided it makes sense to just the Pivotal tooltip for now.
• Add additional EUI components. We have an outstanding list of EUI components which need to be built. (Near term EUI roadmap eui#108)
  • We'll also need improved Jest test coverage (Jest test coverage eui#177).

Branching strategy and workflow

• Kibana master will depend upon EUI branch master
• Kibana 6.x will depend upon EUI branch 6.x
• And so on...

We'll use npm link to test EUI changes in Kibana during development. As we make changes to EUI we'll make corresponding updates in Kibana to consume the changed components, and backport both changes to their respective release branches. This does mean that if you check out an older commit of Kibana, you'll still be consuming the latest commit of the EUI dependency but after discussing this with @epixa, @jimgoodwin, and @snide we decided this will be a rare scenario and is a worthwhile tradeoff for a simpler branching strategy.

Stop using KUI Framework, Bootstrap, and LESS

These are all slated for the chopping block! So as soon as the EUI Framework becomes available within Kibana, new UIs should be built using its components. This includes both React UIs and non-React UIs, because we'll be able to use the EUI CSS classes directly in our markup. Likewise, we should also stop writing new LESS and stop working on the current KUI Framework, which would only add to our technical debt.

Create new components with the Design team

If an engineer needs components which don't exist yet, s/he can enlist the help of the Design team, which will provide designs for new components. They'll then help the engineer implement this component in the EUI Framework or as a Kibana-specific component within Kibana.

Migration plan for third-party plugin developers

• For developers dependent upon KUI, we can provide guidance on how to migrate each KUI component to its closest analogue in EUI.
• For developers dependent upon Bootstrap, we'll suggest they add Bootstrap as an explicit dependency to their plugin.

[cjcenizal]

Depending upon EUI

What's the most appropriate way to branch EUI which both meets Kibana's needs and introduces the least amount of friction to the dev experience?

Branch per Kibana minor

In this case we create branches in EUI which mirror those of Kibana.

Kibana branch	Depends upon EUI branch
master (8.0.0)	master
7.x	7.x

Workflow impact

• Updating Kibana with changes to EUI will be simpler because installing EUI will always bring in the latest changes to that branch
• We'll need to compile and commit the CSS
• Transpiling the JS is time-consuming so Kibana will need to consume the source

Branch per Kibana minor (precompiled)

@spalger suggested this option to avoid consuming untranspiled JS. When we make changes to EUI we can run a task which transpiles the source and pushes it to special sister branches.

Kibana branch	Depends upon EUI branch
master (8.0.0)	master-compiled
7.x	7.x-compiled

Workflow impact

• Simpler consumption of EUI JS
• Slightly more friction when making changes in EUI because we have to manually run a task to make the changes available

Branch per EUI minor

This was suggested by @pugnascotia. In this scenario we consume EUI the same way Cloud does, via the published NPM module.

Kibana branch	Depends upon EUI release	Stemming from EUI branch
master (8.0.0)	2.0.5	2.0.x
7.x	0.1.3	0.1.x

Workflow impact

• Updating Kibana with changes to EUI will be a more complex process. It will require publishing a new patch version in EUI and then upgrading Kibana to use it.
• Consumption of EUI CSS and JS will be simpler and standardized

NOTE: Alternatively, we could depend upon each minor branch instead of the published release. This will incur the same workflow impact as the "Branch per Kibana minor" section, above.

[cjcenizal]

After talking about this with @epixa we decided to depend upon EUI minor branches. Depending upon a branch will make it easier for Kibana engineers to make changes in EUI and then consume them in Kibana. Using the EUI versioning scheme will reduce the number of branches in EUI and simplify the versioning process, though it will make the process of backporting changes more complex.

Because we'll depend upon branches, we need to figure out how to consume the untranspiled source. I like Court's suggestion to publish EUI's build tasks as part of its API, which Kibana can execute as part of its build process to generate the transpiled SCSS and JS. We'll just have to include the build tasks in our CHANGELOG and update version numbers accordingly when we make changes.

[cjcenizal]

@epixa Looks like depending upon branches is problematic because of caching issues. Yarn won't reinstall the dependency, even if the content of the branch changes. I think our best option is to just depend on NPM versions.

[cjcenizal]

Closing this as this process has been mapped out in meta issues and is well underway.