Adding benchmarking script to start monitoring build times

[ouchadam]

Adds a script to run the gradle-profiler so that we can start monitoring and quantifying build time improvements

Usage

• Requires the gradle-profile to be installed
• running from the root of the repository ./tools/benchmark/run_benchmark.sh "comma separated scenario names or blank to run all"`

The warm up and run iterations have been reduced from the defaults, makes the overall suite take less time but at the cost some reliability

Here's some early results...

Recompiling

assemble {
  tasks = [":vector:assembleGPlayDebug"]
}

recompiling without any changes takes 1 second~ it's almost entirely gradle configuration, it shows our gradle setup isn't invaliding the configuration and causing extra processing

Clean builds

assemble clean builds with and without the build cache, build cache is disabled by default

assemble {
  tasks = ["clean", ":vector:assembleGPlayDebug"]
}

assemble_build_cache {
  tasks = ["clean", ":vector:assembleGPlayDebug"]
  gradle-args = ["--build-cache"]
}

From these results we can see that the build cache saves a massive amount of build time, 2m 30s~ down to 14~ seconds, not too unexpected as the build cache doesn't get cleared with a clean and since we're only doing a clean we should get lots of cache hits.

Incremental

Making a non ABI and ABI change in the matrix sdk module

a 15 second~ incremental time shows us at a high level that the SDK module in correctly compiling incrementally, a clean assembly of the sdk module takes 1 minute~. Bencharking ABI changes will be more useful as we introduce more modules as gradle will force clients of modules with ABI to also recompile.

More information can be found at https://developer.android.com/studio/build/profile-your-build

[github-actions]

Unit Test Results

  34 files  ±0    34 suites  ±0   24s ⏱️ ±0s
  73 tests ±0    73 ✔️ ±0  0 💤 ±0  0 ❌ ±0 
180 runs  ±0  180 ✔️ ±0  0 💤 ±0  0 ❌ ±0 

Results for commit b5fadaa. ± Comparison against base commit b5fadaa.

♻️ This comment has been updated with latest results.

[bmarty]

Really interesting thanks.
By monitoring, you mean let the CI run gradle-profiler?

[ouchadam]

@bmarty it could be the CI if we want to automate build times and produce a dashboard (which would be pretty nice!) although to start with running locally periodically would be simpler

[bmarty]

Thanks!