[CI] Build and publish storybooks #87701

brianseeders · 2021-01-07T18:39:30Z

Terraform-managed GCP resources
- Public-facing GCS bucket, ci-artifacts.kibana.dev custom domain and SSL cert, service account and private key generation
- The code for this is stored in a separate repository
- Uploaded files are deleted automatically after 30 days
- Bucket could be used for any publicly-accessible CI artifacts
Adds a ci_composite storybook that automatically includes all of the other storybooks (even if new ones get added)
During CI:
- Create a github commit status for storybook building (both PRs and tracked branches, currently)
- In one task, build all storybooks, including the composite one
- Create index file that links to each storybook
- gzip and upload files to a GCS bucket with a location that's unique for this commit
- Upload the index page to the GCS bucket, at <branch>/latest/index.html, mainly so that tracked branches have a single URL that's always up-to-date
- Complete the github commit status, with a link to the storybook index page
- Add a link to the PR comment for the index page

I'm focused mostly on getting the storybooks building and uploading somewhere in this PR. I expect people who know more about Storybooks will make further improvements (e.g. to the composite) in separate PRs.

Example index page: https://ci-artifacts.kibana.dev/storybooks/pr-87701/6c5dca53b671f41da26d6d73ea536e4f8f2de511/index.html

Link added to the PR comment:

which leads to a page that looks like this:

which links to each individual Storybook, as well as the composite. The link is unique for this commit.

And here's what the checks/statuses look like:

The details link on the finished status links directly to the index page.

smith · 2021-01-11T22:52:45Z

.ci/.storybook/main.js

+
+for(const alias of Object.keys(aliases).filter(a => a !== 'ci_composite')) {
+  // snake_case -> Title Case
+  const title = alias


In a draft PR (#80061) I started by making the aliases an object so that a title could be added. Doing this transformation would probably be fine if there were no title specified, but it would be better to allow a title. For instance here "apm" would end up as "Apm" where we want APM.

smith · 2021-01-11T22:54:27Z

vars/storybooks.groovy

+          <p><a href="${getUrlForCommit()}/composite">Composite Storybook</a></p>
+          <h2>All</h2>
+          <ul>
+            ${listHtml}


Do we need this page? What is the composite storybook missing that this gives us?

I didn't want the links to the individual storybooks to be effectively hidden, so I built this index page to put in front of all of them. This seems particularly useful if you're only concerned about a particular storybook, because the composite takes a long time to load.

smith · 2021-01-11T22:56:35Z

Thanks for kicking this off BTW!

brianseeders · 2021-01-20T19:32:55Z

@elasticmachine merge upstream

spalger

LGTM from an operations perspective

spalger · 2021-02-08T18:10:34Z

vars/storybooks.groovy

+      .collect { it.replace('/', '') }
+      .findAll { it != 'composite' }
+
+    def listHtml = storybooks.collect { """<li><a href="${getUrlForCommit()}/${it}">${it}</a></li>""" }.join("\n")


I'm a tiny bit concerned about creating HTML with string concatenation, though our sources are pretty predictable it would be great if we could limit the character set to /[a-z\-_]/i or something

spalger · 2021-02-08T21:22:13Z

vars/githubPr.groovy

+      ${storybooksMessage}
      ${getDocsChangesLink()}


The newlines that end up in the list here cause the list to render much larger, which would be ideal to work around:

vs

I will get this fixed. I didn't notice before because it only happens if it's not the last link in the list (e.g. when there's a flaky suite)

Ahh, it's actually getDocsChangesLink() that's inserting a blank newline when it's missing. I can still get it fixed in this PR

Yeah, my bad, makes sense. Thank you

brianseeders · 2021-02-10T19:13:26Z

@elasticmachine merge upstream

brianseeders · 2021-02-11T18:10:02Z

@elasticmachine merge upstream

clintandrewhall

LGTM!

…torybooks-ci

brianseeders · 2021-02-18T15:49:36Z

@elasticmachine merge upstream

kibanamachine · 2021-02-18T17:53:48Z

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

💛 Build #107042 was flaky ce4f34d
💚 Build #106146 succeeded 1555c90
💚 Build #105730 succeeded f2e330b
💛 Build #104775 was flaky 4f63800
💚 Build #104359 succeeded ce87f6a

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

kibanamachine · 2021-02-18T19:16:41Z

💔 Backport failed

❌ 7.11: Commit could not be cherrypicked due to conflicts
✅ 7.12 / #91916
✅ 7.x / #91917

Successful backport PRs will be merged automatically after passing CI.

To backport manually, check out the target branch and run:
node scripts/backport --pr 87701

# Conflicts: # src/dev/storybook/aliases.ts

Co-authored-by: Brian Seeders <brian.seeders@elastic.co>

* master: (111 commits) [Logs UI] Replace dependencies in the infra bundle (elastic#91503) [Search Source] Do not request unmapped fields if source filters are provided (elastic#91921) [APM] Kql Search Bar suggests values outside the selected time range (elastic#91918) Refactored component edit policy tests into separate folders and using client integration testing setup (elastic#91657) [Fleet] Don't error on missing package_assets value (elastic#91744) [Lens] Pass used histogram interval to chart (elastic#91370) [Indexpattern management] Use indexPatterns Service instead of savedObjects client (elastic#91839) [Security Solutions] Fixes Cypress tests for indicator match by making the selectors more specific (elastic#91947) [CI] backportrc can skip CI (elastic#91886) Revert "[SOM] fix flaky suites (elastic#91809)" [Fleet] Install Elastic Agent integration by default during setup (elastic#91676) [Fleet] Silently swallow 404 errors when deleting ingest pipelines (elastic#91778) [data.search] Use incrementCounter for search telemetry (elastic#91230) [Fleet] Bootstrap functional test suite (elastic#91898) [Alerts][Docs] Added API documentation for alerts plugin (elastic#91067) Use correct environment in anomaly detection setup link (elastic#91877) [FTSR] Convert to tasks and add jest/api integration suites (elastic#91770) [CI] Build and publish storybooks (elastic#87701) docs: add PHP agent info to docs (elastic#91773) [DOCS] Adds and updates Visualization advanced settings (elastic#91904) ...

brianseeders added 2 commits January 6, 2021 11:21

wip storybooks

889a049

WIP storybook building and uploading

688ef0f

brianseeders added Team:Operations Team label for Operations Team v8.0.0 release_note:skip Skip the PR/issue when compiling release notes Feature:CI Continuous integration labels Jan 7, 2021

brianseeders added 13 commits January 7, 2021 13:50

Switch variable to function

a853aaa

Fix path

de28159

Don't try to set individual object permissions

12409da

Add showInline

c3dc05a

Make storybook assets relative paths

2fccd8a

Build canvas dll

c447a9d

Change canvas asset paths also

9da83af

Build all storybooks

d7d00ad

Add storybooks pr comment link

9cc3944

Use gsutil for upload storybooks assets

922ba2c

Quiet gsutil and add more compressable file types

6c5dca5

Add github commit status for storybooks

b54b34c

Change storybooks context and debug commit status

f442159

brianseeders changed the title ~~[CI] Storybooks~~ [CI] Build Storybooks and upload for public access Jan 11, 2021

brianseeders changed the title ~~[CI] Build Storybooks and upload for public access~~ [CI] Build and publish storybooks Jan 11, 2021

Update context again and add toString() to URL

705049a

brianseeders mentioned this pull request Jan 11, 2021

Build Storybooks on CI #58024

Closed

Build all storybooks again

b891071

smith reviewed Jan 11, 2021

View reviewed changes

brianseeders added 3 commits January 13, 2021 11:57

Re-enable full pipeline

5b1ed96

Merge remote-tracking branch 'upstream/master' into storybooks-ci

da8a193

Limit storybook webpack parallelism in CI, and enable caching

dbd0574

spalger approved these changes Feb 8, 2021

View reviewed changes

spalger reviewed Feb 8, 2021

View reviewed changes

Merge branch 'master' into storybooks-ci

f2e330b

Merge branch 'master' into storybooks-ci

1555c90

clintandrewhall approved these changes Feb 12, 2021

View reviewed changes

brianseeders added 2 commits February 16, 2021 17:09

Merge remote-tracking branch 'upstream/master' into storybooks-ci

350a292

Merge branch 'storybooks-ci' of github.com:brianseeders/kibana into s…

ce4f34d

…torybooks-ci

Merge branch 'master' into storybooks-ci

6752c65

brianseeders added v7.12.0 v7.13.0 v7.11.2 auto-backport Deprecated - use backport:version if exact versions are needed labels Feb 18, 2021

brianseeders merged commit 03206b6 into elastic:master Feb 18, 2021

brianseeders deleted the storybooks-ci branch February 18, 2021 19:13

kibanamachine mentioned this pull request Feb 18, 2021

[7.12] [CI] Build and publish storybooks (#87701) #91916

Merged

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Feb 18, 2021

[CI] Build and publish storybooks (elastic#87701)

bfab2a3

kibanamachine mentioned this pull request Feb 18, 2021

[7.x] [CI] Build and publish storybooks (#87701) #91917

Merged

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Feb 18, 2021

[CI] Build and publish storybooks (elastic#87701)

119d266

brianseeders mentioned this pull request Feb 18, 2021

[7.11] [CI] Build and publish storybooks (#87701) #91919

Merged

brianseeders added a commit to brianseeders/kibana that referenced this pull request Feb 18, 2021

[CI] Build and publish storybooks (elastic#87701)

809415f

# Conflicts: # src/dev/storybook/aliases.ts

kibanamachine added a commit that referenced this pull request Feb 18, 2021

[CI] Build and publish storybooks (#87701) (#91917)

854e833

Co-authored-by: Brian Seeders <brian.seeders@elastic.co>

kibanamachine added a commit that referenced this pull request Feb 18, 2021

[CI] Build and publish storybooks (#87701) (#91916)

2ec7989

Co-authored-by: Brian Seeders <brian.seeders@elastic.co>

brianseeders added a commit that referenced this pull request Feb 19, 2021

[7.11] [CI] Build and publish storybooks (#87701) (#91919)

2b9763e

jbudz mentioned this pull request Jan 24, 2022

[ci] Build and publish webpack bundle reports #123659

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[CI] Build and publish storybooks #87701

[CI] Build and publish storybooks #87701

brianseeders commented Jan 7, 2021 •

edited

Loading

smith Jan 11, 2021

smith Jan 11, 2021

brianseeders Jan 13, 2021

smith commented Jan 11, 2021

brianseeders commented Jan 20, 2021

spalger left a comment

spalger Feb 8, 2021

spalger Feb 8, 2021

brianseeders Feb 8, 2021

brianseeders Feb 8, 2021

spalger Feb 8, 2021

brianseeders commented Feb 10, 2021

brianseeders commented Feb 11, 2021

clintandrewhall left a comment

brianseeders commented Feb 18, 2021

kibanamachine commented Feb 18, 2021

kibanamachine commented Feb 18, 2021

[CI] Build and publish storybooks #87701

[CI] Build and publish storybooks #87701

Conversation

brianseeders commented Jan 7, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

smith commented Jan 11, 2021

brianseeders commented Jan 20, 2021

spalger left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

brianseeders commented Feb 10, 2021

brianseeders commented Feb 11, 2021

clintandrewhall left a comment

Choose a reason for hiding this comment

brianseeders commented Feb 18, 2021

kibanamachine commented Feb 18, 2021

💚 Build Succeeded

Metrics [docs]

History

kibanamachine commented Feb 18, 2021

💔 Backport failed

brianseeders commented Jan 7, 2021 •

edited

Loading