Respect -C opt-level=s to build smaller artifacts

[nazar-pc]

We're running out of space in CI and wasm-opt-sys is a big contributor to that. I'm wondering if it would be possible to respect -C opt-level=s to build smaller binaries without users being aware of OPT_LEVEL environment variable.

If that is already the case then consider cleaning up things that are no longer needed at the end of the build step to reduce the total space used by build artifacts.

[brson]

I believe the C++ build of wasm-opt already respects -C opt-level=s, at least when the optimization level is set by cargo (I am not clear on what happens if it is set by RUSTFLAGS).

I have not verified this though.

The C++ build is handled by the cc crate which generally tries to make the C++ optimization settings match the Rust optimization settings, including setting the correct C++ compiler args based on cargo's OPT_LEVEL env var.

To affect only the C++ it should also be possible to override CXXFLAGS.

[nazar-pc]

Okay, what about potentially deleting build artifacts then? I don't think it makes a lot of sense to keep anything except final binaries/libraries here for the purposes of the crate (I can create a separate issue for that).

[brson]

@nazar-pc I am reluctant to try to delete intermediate build artifacts as part of the build process itself, mostly because it strikes me as an atypical thing for a build script (or any build system) to do. There are no pre-existing facilities to help with this either - I would just have to use my judgment about what to delete out of the target directory.

I don't think this crate is generating an abnormal quantity of build artifacts. On my local machine the entire source and debug build size is 3.5 GB, which doesn't seem unusual for a Rust project.

[nazar-pc]

I am reluctant to try to delete intermediate build artifacts as part of the build process itself, mostly because it strikes me as an atypical thing for a build script (or any build system) to do.

I don't think this is necessarily the case for nested dependencies. When I install this crate from crates.io, it'll be compiled once and then just used whenever necessary. So it makes sense to produce necessary artifacts and remove everything else as the likelihood of it being necessary later is low.

I don't think this crate is generating an abnormal quantity of build artifacts.

I'm primarily talking about opt directory, which is under full control of the crate, it must know what is stored there and what is safe to delete.

In my case of debug build wasm-opt-sys-xyz/out is 408M and then .rlib file which is 218M in size on to of that (this is what I think is actually needed, those 408M must be safe to remove). This is worsened by the fact that in Substrate (that recently switched to this crate) every crate is built in its independent workspace, which means with 4 runtimes right now we have all that duplicated 4 times, so this is over 1.6G of wasted space for a relatively small utility. Out of 10G available in GitHub Actions CI that is non-negligible amount and actually caused our CI to run out of space already. There are Substrate-based projects with 6+ runtimes.

This is why I think removing at least part of those temporary files would be really beneficial.

[brson]

I am interested to hear more from the substrate authors about this, as it does seem to be primarily an issue with how the substrate build is orchestrated. The issue I know to follow is this one, paritytech/substrate#12672, which hasn't been updated recently.

I would also be interested in evidence that this is affecting other substrate chains.

My first inclination, knowing that every substrate runtime is built in its own workspace, is to clean those builds after they are done; or to otherwise be more efficient in building substrate runtimes.

Even if making the wasm-opt build more efficient will help for this specific case in the near term, it seems like this probably will probably happen again with substrate runtimes.

If I were to add an ability to delete intermediate files I would probably make it opt-in with an environment variable. The intermediate files are evenly split between object files and the archive files that collects them. The object files are easy to delete. The archive less so, as that is the output that is passed back to cargo to link into the rlib. They could possibly be deleted in the wasm-op-cxx-sys build script, which runs after wasm-opt-sys, but that's getting very hacky, and I don't know what mechanisms might be available to help one build script discover the output dir of another crate.

[brson]

Here's another issue as linked in the substrate PR: paritytech/substrate#12671

[nazar-pc]

I think the fewer bytes crate needs on disk the better all other things being equal. At least object files should be safe to remove as an intermediate build product. Archive files will likely need to stay.