Most of the statements made in this doc correspond to the 1.0.0 version. As of now (Late Fall '23), it's an essay describing the process of building the scalac-profiling tool.
The goal of this proposal is to allow Scala developers to optimize their codebase to reduce compile times, spotting inefficient implicit searches, expanded macro code, and other reasons that slow down compile times and decrease developer productivity.
This repository holds the compiler plugin and a fork of mainstream scalac that will be eventually be merged upstream. This work is prompted by Morgan Stanley's proposal and was approved in our last advisory board.
The required changes to the compiler, Scalac, are the following:
- Collect all statistics and optimize checks.
- Initialize statistics per global.
- Add extra timers and counters.
Effective since Scala 2.12.5, all of those changes have been released.
The project uses a forked scalac version that is used to compile both the compiler plugin and several OSS projects from the community. The integration tests are for now Better Files and Wartremover, and they help us look into big profiling numbers and detect hot spots and misbehaviours.
If you think a particular codebase is a good candidate to become an integration test, please open an issue.
The proposal is divided into three main areas:
- Data generation and capture.
- Data visualisation and comparison.
- Reproducibility.
How to tackle each of these problems to make the implementation successful?
The generation of data comes from the guts of the compiler. To optimize for impact, the collection of information is done in two different places (a compiler plugin and a forked scalac).
- A compiler plugin to get information from the macro infrastructure independently of the used Scalac version.
- Profiledb readers and writers to allow IDEs and editors to read and write profiledb's.
- An sbt plugin for reproducibility that warms up the compiler before profiling.
The work is split into two parts so that Scala developers that are stuck in previous Scala versions can use the compiler plugin to get some profiling information about macros.
This structure is more practical because it allow us to evolve things faster in the compiler plugin, or put there things that cannot be merged upstream.
The profiling data will be accessible in two different ways (provided that the pertinent profiling flags are enabled):
- A summary of the stats will be printed out in every compile run.
- A protobuf file will be generated at the root of the class files directory.
- The file is generated via protobuf so that it's backwards and forwards binary compatible
- The protobuf file will contain all the profiling information.
Why a protobuf file instead of a JSON file? Forwards and backwards binary compatibility is important -- we want our tooling to be able to read files generated by previous or upcoming versions of the compiler. Our goal is to create a single tool that all IDEs and third-party tools use to parse and interpret the statistics from JARs and compile runs.
We're collaborating with Intellij to provide some of the statistics within the IDE (e.g. macro invocations or implicit searches per line). We have some ideas to show this information as heat map in the future.
Getting reproducible numbers is important to reason about the code and identifying when a commit increases or decreases compile times with certainty.
To do so, several conditions must be met: the compiler must be warmed up, the load in the running computer must be low, and the hardware must be tweaked to disable options that make executions non reproducible (like Turbo Boost).
However, this warming up cannot be done in an isolated scenario as Scalac's benchmarking infrastructure does because it doesn't measure the overhead of the build tool calling the compiler, which can be significant (e.g. in sbt).
As a result, reproducibility must be achieved in the build tool itself. The goal of this project is to provide an sbt plugin that warms up the compiler by a configurable amount of time. It also bundles recommendations and tips on how and where to run compilation.
In the following sections, I elaborate on the collected data that we want to extract from the compiler as well as technical details for every section in the original proposal.
Per call-site, file and total:
- How many macros are expanded?
- How long do they take to run?
- How many tree nodes do macros create?
Getting hold of this information requires changes in mainstream scalac.
Per call-site, file and total:
- How many implicit searches are triggered per position?
- How many implicit searches are triggered for a given type?
- How long implicit searches take to run?
- How many implicit search failures are?
- How many implicit search hits are?
- What's the ratio of search failures/hits?
These are the requirements that the proposal lays out.
Note that in some cases, this plugin provides more information than the requested by the original proposal.
- Compilation time totally (this is provided by
-Ystatistics) - Macro details
- Time per file
- Time per macro
- Invocations
- Per type
- Total time
- Flamegraph of all macros
- Implicit search details (time and number)
- By type
- By invocation (only number for now)
- By file (can be aggregated from the "by invocation" data)
- Flamegraph of all the implicit searches
- User time, kernel time, wall clock, I/O time.
This feature was already provided by Scalac, implemented in this PR. - Time for flagged features (for certain features – e.g. optimisation)
- The best way to capture this information is running statistics for the compiler with and without optimization, and compare the profiles. There are also some extra counters.
- Time resolving types from classpath
- Total