Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

38x performance regression in lewton since rust 1.26 #55446

Closed
est31 opened this issue Oct 28, 2018 · 1 comment
Closed

38x performance regression in lewton since rust 1.26 #55446

est31 opened this issue Oct 28, 2018 · 1 comment
Labels
C-bug Category: This is a bug. I-slow Issue: Problems and improvements with respect to performance of generated code. regression-from-stable-to-stable Performance or correctness regression from one stable version to another. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. WG-llvm Working group: LLVM backend code generation

Comments

@est31
Copy link
Member

est31 commented Oct 28, 2018

Decoding the test assets in the benchmark suite of lewton takes 2.5449s if compile with rustc 1.25.0 and 96.5645s with rustc 1.26.0. This is a 38 fold regression.

To reproduce:

$ git clone https://github.com/RustAudio/lewton/
$ cd lewton/dev/cmp
$ RUSTFLAGS="-C codegen-units=1" cargo +1.25.0 run bench --release
    Finished release [optimized] target(s) in 0.0 secs
     Running `target/release/cmp bench`

File bwv_1043_vivace.ogg has matching hash inside hash list, skipping download
File bwv_543_fuge.ogg has matching hash inside hash list, skipping download
File maple_leaf_rag.ogg has matching hash inside hash list, skipping download
File hoelle_rache.ogg has matching hash inside hash list, skipping download
File thingy-floor0.ogg has matching hash inside hash list, skipping download
File audio_simple_err.ogg has matching hash inside hash list, skipping download

Comparing speed for bwv_1043_vivace.ogg : libvorbis=0.4688s we=0.6376s difference=1.36x
Comparing speed for bwv_543_fuge.ogg    : libvorbis=0.6750s we=0.9988s difference=1.48x
Comparing speed for maple_leaf_rag.ogg  : libvorbis=0.1916s we=0.2751s difference=1.44x
Comparing speed for hoelle_rache.ogg    : libvorbis=0.3393s we=0.4721s difference=1.39x
Comparing speed for thingy-floor0.ogg   : libvorbis=0.1525s we=0.1699s difference=1.11x

Overall time spent for decoding by libvorbis: 1.8272s
Overall time spent for decoding by us: 2.5536s
Overall ratio of difference: 1.40x
$ RUSTFLAGS="-C codegen-units=1" cargo +1.26.0 run bench --release
    Finished dev [unoptimized + debuginfo] target(s) in 0.0 secs
     Running `target/debug/cmp bench --release`

File bwv_1043_vivace.ogg has matching hash inside hash list, skipping download
File bwv_543_fuge.ogg has matching hash inside hash list, skipping download
File maple_leaf_rag.ogg has matching hash inside hash list, skipping download
File hoelle_rache.ogg has matching hash inside hash list, skipping download
File thingy-floor0.ogg has matching hash inside hash list, skipping download
File audio_simple_err.ogg has matching hash inside hash list, skipping download

Comparing speed for bwv_1043_vivace.ogg : libvorbis=1.7041s we=22.9206s difference=13.45x
Comparing speed for bwv_543_fuge.ogg    : libvorbis=2.7188s we=38.3622s difference=14.11x
Comparing speed for maple_leaf_rag.ogg  : libvorbis=0.8191s we=11.1756s difference=13.64x
Comparing speed for hoelle_rache.ogg    : libvorbis=1.3928s we=17.8183s difference=12.79x
Comparing speed for thingy-floor0.ogg   : libvorbis=0.9074s we=6.2878s difference=6.93x

Overall time spent for decoding by libvorbis: 7.5422s
Overall time spent for decoding by us: 96.5645s
Overall ratio of difference: 12.80x

The same behaviour also occurs without the RUSTFLAGS env var but I've added it here to indicate that this bug is a different issue from #53833 or #47745.

Note that the bug is not fixed in stable 1.30.0 (da5f414c2 2018-10-24) or 1.31.0-nightly (cae6efc37 2018-10-27) (latest nightly as of filing this bug): On stable, the benchmark takes 80.0889s, on nightly, it takes 77.7243s.

@kennytm kennytm added I-slow Issue: Problems and improvements with respect to performance of generated code. regression-from-stable-to-stable Performance or correctness regression from one stable version to another. C-bug Category: This is a bug. WG-llvm Working group: LLVM backend code generation labels Oct 28, 2018
@Mark-Simulacrum Mark-Simulacrum added the T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. label Oct 28, 2018
@est31
Copy link
Member Author

est31 commented Oct 28, 2018

Ohhhh I see it. Of course.

Running target/debug/cmp bench --release

Running target/release/cmp bench

This seems to be a weird regression in cargo that made cargo run bench --release being treated differently between the two releases, somehow ignoring the --release flag after 1.26.0.

Closing lol I'm stupid.

@est31 est31 closed this as completed Oct 28, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-bug Category: This is a bug. I-slow Issue: Problems and improvements with respect to performance of generated code. regression-from-stable-to-stable Performance or correctness regression from one stable version to another. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. WG-llvm Working group: LLVM backend code generation
Projects
None yet
Development

No branches or pull requests

3 participants