Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decrease the amount of time performance testing takes in GitHub Actions #44679

Open
desrosj opened this issue Oct 4, 2022 · 3 comments
Open
Labels
[Type] Automated Testing Testing infrastructure changes impacting the execution of end-to-end (E2E) and/or unit tests. [Type] Performance Related to performance efforts

Comments

@desrosj
Copy link
Contributor

desrosj commented Oct 4, 2022

What problem does this address?

The Performance Tests workflow is the longest running test workflow by far, taking anywhere from 45 minutes to 2 hours.

Let's look at ways to speed this up.

What is your proposed solution?

It seems there are two groups of performance tests that currently run: post-editor and site-editor. If these could be run in parallel, that would potentially cut the time to run in half.

@desrosj
Copy link
Contributor Author

desrosj commented Oct 4, 2022

Another aspect to think about. The workflow is required to complete for a PR to be allowed to merge. However, this workflow should not necessarily always be required. For example, if only .github/workflow/*.yml files are edited, then this isn't really needed (unless the actual performance.yml file is updated).

I'm sure there are other scenarios where this could be skipped entirely as well.

@Mamaduka Mamaduka added [Type] Automated Testing Testing infrastructure changes impacting the execution of end-to-end (E2E) and/or unit tests. [Type] Performance Related to performance efforts labels Oct 4, 2022
@Mamaduka
Copy link
Member

Mamaduka commented Oct 5, 2022

We can also skip or make the action optional for the docs directory. The changes there can't affect the Editor(s) performance.

@desrosj
Copy link
Contributor Author

desrosj commented Oct 6, 2022

Another observation I had today as it relates to caching.

The performance testing runs npm install multiple times to ensure that the needed dependencies and versions are installed each time a new branch is compared. But, the same cache key is used for the performance workflow as all other workflows. It's possible that the caching benefits are not being experienced in this situation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[Type] Automated Testing Testing infrastructure changes impacting the execution of end-to-end (E2E) and/or unit tests. [Type] Performance Related to performance efforts
Projects
None yet
Development

No branches or pull requests

2 participants