rustc: Fix regression where jemalloc isn't used

[alexcrichton]

In #56986 the linkage of jemalloc to the compiler was switched from the
driver library to the rustc binary to ensure that only rustc itself uses
jemalloc. In doing so, however, it turns out jemalloc wasn't actually
linked in at all! None of the symbols were referenced so the static
library wasn't used. This means that jemalloc wasn't pulled in at all.

This commit performs a bit of a dance to reference jemalloc symbols,
attempting to pull it in despite LLVM's optimizations.

Closes #57115

[rust-highfive]

r? @nikomatsakis

(rust_highfive has picked a reviewer for you, use r? to override)

[alexcrichton]

@bors: try

[bors]

⌛ Trying commit 5265c31 with merge 3550c27...

[bors]

☀️ Test successful - status-travis
State: approved= try=True

[alexcrichton]

@rust-timer build 3550c27

[rust-timer]

Success: Queued 3550c27 with parent ec19464, comparison URL.

[nikomatsakis]

So my take on this:

• The code itself seems "obviously" "fine" (as in, not going go to break things)
• But oh god what a hack

It seems surprising to me that there is no more elegant way to do this.

I'm willing therefore to r+ though I might prefer if somebody else who was closer to the whole linking story did so, since maybe they will have another idea. (e.g., @petrochenkov is asking good questions here).

[nikomatsakis]

Also, the try build is ready, not sure what use you had in mind, @alexcrichton ?

[alexcrichton]

The most elegant way to do this without looking like a hack is probably to use linker flags directly, but actually doing that in Rust would look like more of a hack unfortunately. We don't have that level of control over how things are built to know what's where on the command line, so I've found symbol references to be much more flexible.

Also sorry should have mentioned, but I'd like to do a perf run to confirm that this actually fixes the regression this claims to fix.

[alexcrichton]

(I'd also like to add more comments in the code about the hacks here code-wise, was just waiting on perf)

[rust-timer]

Finished benchmarking try commit 3550c27

[alexcrichton]

Yay regression is indeed fixed!

I've added some comments, and perhaps in addition to @petrochenkov @michaelwoerister may want to take a look?

[petrochenkov]

Other crates that want to use static jemalloc will have to do something like this as well?

[alexcrichton]

In theory, yes, but in practice it isn't much of a problem. Almost all Rust code that isn't rustc is built with all Rust code statically linked, so the references to the jemalloc symbols will come through things like libstd or just bland Vec::push

[petrochenkov]

@alexcrichton

I think the --whole-archive with -l vs #[link] is probably a bug actually?

Some update: apparently I misremembered the details, command line and attribute behave identically (that's good).

The difference was that static uses whole-archive, but static-nobundle doesn't.
That's also not entirely correct since bundling and whole-archive-ness are somewhat independent.
For example, for linking jemalloc in this case we'd want exactly nobundle + whole-archive.

[alexcrichton]

From my experience at least creating a composable system where libraires can specify linkage and whatnot is insanely hard to say the least and likely downright impossible. We do our best when we can and otherwise just try to fix things when a bug comes up. We're definitely not consistent though :(

[nikomatsakis]

@bors r+

Since it seems like @petrochenkov doesn't have any blocking objections, I'm going to r+ here for now. Feel free to r- or r? someone else though.

[bors]

📌 Commit ccbf28e has been approved by nikomatsakis

[bors]

⌛ Testing commit ccbf28e with merge af2c159...

[bors]

☀️ Test successful - status-appveyor, status-travis
Approved by: nikomatsakis
Pushing af2c159 to master...