Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CI Benchmarks for Browser environments #3136

Open
maschad opened this issue Sep 9, 2024 · 3 comments
Open

Add CI Benchmarks for Browser environments #3136

maschad opened this issue Sep 9, 2024 · 3 comments
Assignees
Labels
chore Issue is a chore

Comments

@maschad
Copy link
Member

maschad commented Sep 9, 2024

#3084 seeks to introduce benchmarks on CI for local nodes in a Node.js environment. These are intended to be run per PR workflow and currently take ~ 12 mins.

We would like to support running these in the browser on CI but that would require extra setup which could take the workflow in excess of ~25 mins. I don't think this would be appropriate to be run per PR or even in a release process, but could be useful post-release (the downside of that is if we introduce a regression in the browser it would be caught after the release)

I do think though that there is a low probability one could introduce a performance regression that is vastly more significant in the browser than in node, but still possible.

Copy link
Member

Okay, 25+ minutes seems like a no-go. It's too much. Why does it take so long?

@maschad
Copy link
Member Author

maschad commented Sep 11, 2024

I elaborated a bit more here but TLDR If we can essentially cache the SDK builds + pretest, as well as the chrome setup ubuntu builds, we can take each benchmark time down to roughly 6 mins which would be 12 mins total.

@maschad
Copy link
Member Author

maschad commented Sep 11, 2024

Something to note as well CodSpeedHQ/action#107

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
chore Issue is a chore
Projects
None yet
Development

No branches or pull requests

3 participants