elfx86exts 0.4.3 / capstone 0.7.0 overlooks instructions in some circumstances

[rowanworth]

Basically I have two builds of elfx86exts, both of which I'm running against the same large binary (126M). Both builds were created via cargo install elfx86exts with rust-1.48.0. The first build is stock elfx86exts-0.4.3 and the second build is one I patched manually to update the capstone dependency to ^0.10 (previous build was against the capstone-0.7.0 crate).

The two behave very differently. Here's the stock version:

$ time elfx86exts test_binary
MODE64 (call)
SSE1 (stmxcsr)
AVX (vzeroupper)
AVX2 (vpcmpeqd)
NOVLX (vpcmpeqd)
CPU Generation: Unknown

real	0m0.044s
user	0m0.007s
sys	0m0.015s

And here's my manually patched version:

$ time elfx86exts test_binary
MODE64 (call)
SSE1 (stmxcsr)
AVX (vzeroupper)
AVX2 (vpcmpeqd)
NOVLX (vpcmpeqd)
AVX512 (vmovdqu32)
SSE2 (pcmpeqd)
CMOV (cmovne)
VLX (vpbroadcastd)
BWI (vpcmpeqb)
BMI (andn)
CDI (vptestnmd)
DQI (kmovb)
BMI2 (shlx)
SSE3 (movddup)
SSE41 (roundss)
MMX (emms)
SSE42 (pcmpistri)
SSSE3 (phaddd)
NOT64BITMODE (xchg)
CPU Generation: Unknown

real	0m47.405s
user	0m29.692s
sys	0m17.684s

Wildly different runtimes and detected instruction sets. It seems like the stock version is exiting early but it does so silently and without raising any error (exit code is 0).

I had to change two lines to get here; the first was in Cargo.tml:

 [dependencies.capstone]
-version = "^0.7"
+version = "^0.10"

And the second in src/main.rs:

             for group_code in detail.groups() {
-                if seen_groups.insert(group_code) {
+                if seen_groups.insert(group_code.clone()) {
                     // If insert returned true, we hadn't seen this code before.
                     if let Some(desc) = describe_group(group_code.0) {

To satisfy the borrow checker -- I guess the return type of detail.groups() has changed upstream but I didn't investigate in detail, just picked up the biggest hammer from my rust beginner's toolkit :)

[pkgw]

Hm, that is a pretty substantial difference in the output! And I presume that we have good reason to believe that the output from the capstone 0.10 series is more accurate.

Fortunately the main branch of the repo has already been bumped to capstone 0.10, so all I need to do is issue a new release. I'll do that now — I have some nice release automation so it's easy.

[pkgw]

OK, version 0.5.0 is now released, and should hopefully fix your issue. I'll close this but feel free to comment or file a separate issue if there's more to discuss.

[rowanworth]

The binary was compiled for AVX512 and objdump -d on the same binary reports a lot of vmovdqu instructions so yes I'm confident the new results are more accurate.

I do notice it requires an absurd amount of memory (42GB of RAM to analyse this 125MB binary), but also that could be related to my .clone() hack? Tried to build v0.5.0 to verify but note it doesn't compile on cargo's stable channel:

  Downloaded object v0.27.0
error: failed to compile `elfx86exts v0.5.0`, intermediate artifacts can be found at `/localData/000scratch/cargo-installCifRJZ`

Caused by:
  failed to parse manifest at `/d/home/dug/rowanw/.cargo/registry/src/gitpro.ttaallkk.top-1ecc6299db9ec823/object-0.27.0/Cargo.toml`

Caused by:
  feature `resolver` is required

  this Cargo does not support nightly features, but if you
  switch to nightly channel you can add
  `cargo-features = ["resolver"]` to enable this feature

If I revert to object-0.21.1 in elfx86exts-0.5.0/Cargo.toml the build succeeds and the required memory remains 42GB¹. Not sure if that's elfx86exts iself or some other component (capstone?) using the RAM, but objdump -d uses 120MB RAM so I don't think there's an excuse for using 350x that :)

¹ as measured by /bin/time: 28.11user 16.39system 0:44.54elapsed 99%CPU (0avgtext+0avgdata 44040892maxresident)k

[pkgw]

That's odd — I'm definitely building this package using stable Rust, and object 0.27 doesn't cause any problems for me.

As for the memory consumption, I hope that it's only apparent, and not actually filling up gigabytes of in-memory buffers. But I'm not sure. I am pretty confident, though, that this program is operating sensibly — if the lower-level libraries are allocating huge buffers I'm not sure if we'll be able to do anything about that :-/

[rowanworth]

It's definitely using gigabytes of memory; it sent my workstation (w/ 32GB RAM) into swap every time I tried to run it and I moved to a machine with 128GB RAM to get those timings :)

I wonder if the rust version contributes to the resolver issue? object-0.27.0/Cargo.toml has this:

[package]
...
resolver = "2"

And according to https://doc.rust-lang.org/cargo/reference/resolver.html

The version "1" resolver is the original resolver that shipped with Cargo up to version 1.50. The default is "2" if the root package specifies edition = "2021" or a newer edition. Otherwise the default is "1".

I'm on rustc/cargo 1.48.0 so that probably explains it.

[pkgw]

Wow! I wonder where that massive memory allocation is coming from — I can't imagine that it's needed for the kind of analysis this tool is doing. Maybe the default disassembly computes some kind of massive metadata structures, and we can avoid that somehow? To be honest, I don't really use this program anymore and I don't have the bandwidth to dig into its bugs, but I would love for someone to figure out what's going on here!

[rowanworth]

Yeah fair enough! I spent 10 minutes looking into capstone before getting dragged into more pressing matters. Thanks for the tool and the object-0.27.1 fix :)

[Ristovski]

@pkgw I can confirm that the massive memory allocation is still the case as of the latest release. Analyzing a 123MB binary OOMs my 24GB RAM machine.

[oldherl]

I can confirm that the massive memory allocation is still the case as of the latest master branch. Analyzing a 17MB binary (/bin/supertuxkart) OOMs my 16GB RAM machine.

[pkgw]

I don't use this tool myself anymore and unfortunately I'm not in a position to spend time on it, but maybe the proposal in #47 would help with addressing this.

[pkgw]

Oh, also, the original issue was something different, so I'm opening a new issue for the memory consumption problem. Please continue discussion in #90.