Remove fastcomp

[kripken]

Almost a year ago, in last July we declared the upstream LLVM backend ready for testing, described its benefits (such as faster compile times and smaller code), and said that we’d like to eventually remove fastcomp. After that, in version 1.39.0, which was released on October 18 last year, we switched the default to the upstream LLVM backend. It has been around 8 months since then, and after a lot of work we’ve all been doing (thanks to everyone that helped with issues and PRs!) we are not aware of any remaining things that should block people from using the upstream backend, and so we’ll soon (in the next release) start showing a deprecation warning when using fastcomp, as @sbc100 mentioned on the mailing list earlier

We’d now like to make concrete plans for removing fastcomp from the master branch, including a timeline. “Removing fastcomp” means that in new releases after the cutoff point only the upstream LLVM backend will be available. That is, as always

emsdk install latest

will install the latest version using the upstream backend (which has been the default for a while as mentioned earlier). What will change is that, after we remove fastcomp, you won’t be able to do this:

emsdk install latest-fastcomp

That is, there will be no fastcomp version of “latest”. You will still be able to use older versions that we released with fastcomp, for example

emsdk install 1.39.10-fastcomp

So you can keep on using fastcomp with those older versions if you absolutely have to - but we strongly recommend upgrading to upstream since it has significantly better code size and compile times.

All of this is what will change once fastcomp is removed from master, so it isn’t happening yet. But we’d like to do it soon, since we’ve already been focusing all new development on the upstream backend as much as possible. Removing fastcomp will let us do so much more efficiently since we won’t need to make sure every commit works in both backends. As a result we’ll be able to get new code size improvements, support for new wasm features like native exceptions support and SIMD, etc., more quickly!

Please let us know if you have any concerns about removing fastcomp, bugs that we are not aware of or that haven’t been prioritized properly, etc.

[kripken]

Ok, it's been over 2 weeks so I think we are ready to move forward here. I suggest the following next steps:

1. Figure out if there is anything we want to get into the last release that supports fastcomp. I know @juj has some PRs there.
2. Once that is in shape, the next release we tag after that is 1.40.0 - @sbc100 I think this is what you've suggested?
3. We wait a while to see if there are any issues, both in general and in the fastcomp build. We may tag 1.40.1 etc. during this time, if necessary.
4. After we are sure 1.40.0 (or maybe 1.40.1 or such if we had to tag more) are ok as the last fastcomp build, then and only then we can start to remove fastcomp in trunk.
5. The next time we tag after that, we call it 2.0.0. (This may be a good time to discuss switching to semantic versioning, but that's separate from this issue.)

Thoughts?

[sbc100]

Basically agree with all this.

Re step (5) I propose we do the smallest possible change to remove fastcomp support and then tag 2.0.0 right away. That way the start of the 2.0.X series and the beginning of the 1.40.X will be almost identical in terms that actual code they contains. 2.0.1 can then be the first one with large changes in it.

[sbc100]

Regarding semantic versioning, I'm going to be on the against side of that argument I think :)

[kripken]

Sounds good about (5). In fact we can do no change at all in emscripten and just update the emsdk to not report builds for -fastcomp for the new versions, and tag that.

[vbora]

Our code base does not build with the new backend. It only build with fastcomp. Can we request not removing fastcomp support for now?

[kripken]

@vbora Please provide more information about what does not work - if there are problems, we'd like to fix them! Specific bug reports with error messages and testcases would be great.

Note that even after fastcomp is removed you will be able to use fastcomp, it just won't get new updates. That is, it will be removed from new development, but also existing builds will work as they always did. So you will be able to keep using fastcomp as long as you want - but we'd recommend upgrading to get the benefits of upstream (much faster compilation, smaller code size, new wasm features), of course.

[vbora]

@kripken Thanks for a quick reply.

We too would like to have the option to upgrade emscripten as required.

I do not have a handy testcase I can share, but here's our setup. We build a static openssl library in linux and use that static library in our build. When using the latest emscripten build, we get these errors. I looked up possible solutions for these, but could not come up with anything that worked, except using the old fastcomp backend.

One more peculiar thing about our build is that we currently build with -s WASM=0 -s LEGACY_VM_SUPPORT=1 as we need our webapp to run on IE.

wasm-ld: error: s23_clnt_1a0e12fc.o: machine type must be wasm32
wasm-ld: error: ssl_lib_aef6f151.o: machine type must be wasm32
wasm-ld: error: ssl_err2_d2fbbbf1.o: machine type must be wasm32
wasm-ld: error: ssl_ciph_63c44977.o: machine type must be wasm32
wasm-ld: error: ssl_stat_477f6394.o: machine type must be wasm32
wasm-ld: error: ssl_algs_067c1c5a.o: machine type must be wasm32

wasm-ld: error:
ex_data_1eb8e71b.o: machine type must be wasm32
wasm-ld: error: obj_dat_228a80cd.o: machine type must be wasm32
wasm-ld: error: hmac_55711091.o: machine type must be wasm32
wasm-ld: error: bn_lib_90d846b1.o: machine type must be wasm32
wasm-ld: error: rsa_lib_02f3ad0f.o: machine type must be wasm32
wasm-ld: error: rsa_crpt_505558d1.o: machine type must be wasm32
wasm-ld: error: bio_lib_0896864c.o: machine type must be wasm32
wasm-ld: error: bss_mem_f1d9ca76.o: machine type must be wasm32
wasm-ld: error: rand_lib_75c76d4f.o: machine type must be wasm32
wasm-ld: error: err_21ca39e7.o: machine type must be wasm32
wasm-ld: error: digest_da40837e.o: machine type must be wasm32
wasm-ld: error: evp_enc_d2867091.o: machine type must be wasm32
wasm-ld: error: e_rc4_172aac8b.o: machine type must be wasm32
wasm-ld: error: m_md4_dc18baa3.o: machine type must be wasm32
wasm-ld: error: too many errors emitted, stopping now (use -error-limit=0 to see all errors)

[sbc100]

The problem here is that your libraries all need to be re-built with the upstream compiler.

All those object files look like they were compiled with fastcomp still.

[kripken]

Yes, a full-rebuild of all those should fix those errors as @sbc100 said.

And -s WASM=0 -s LEGACY_VM_SUPPORT=1 are fully supported in upstream, so those should just work too.

[vbora]

@kripken @sbc100 We were building openssl to created static libraries using gcc and link them with our emscripten code. Is fastcomp better able to handle static library built by gcc than upstream?

Another thing I was exploring was to build openssl using emscripten directly. I found this , but the link inside it with instructions is not valid any more. Any suggestions for it?

[kripken]

@vbora there is no difference between fastcomp and upstream on static libraries built by gcc - neither compiler can link with those. wasm/asm.js can only link with other wasm/asm.js, not x86 or ARM etc.

Is it possible those static libraries were built not by gcc but by emcc? That would work, and rebuilding those with new emcc (with upstream) will work too.

[kripken]

Ok, I think we are ready to move forward here! That means, as discussed earlier,

• Next tagged version will be 1.40.0.
• We give that some time to see if any issues happen. No fastcomp removals yet.
• When it's time to tag again, if no issues happened, we can update the emsdk to report no fastcomp builds are available for 2.0.0 and onwards, and tag 2.0.0.
• We can then start to remove parts of fastcomp in the emscripten repo.

[kripken]

1.40.0 has been tagged, emscripten-core/emsdk#577

[sbc100]

This is now done!

I'll create a followup issue to coordinate the cleanup