gnugrep: enable parallel build and tests

[trofi]

On a 16-core system number changes are:

• before: 1m34s
• after: 39s

2.5x speedup. If ./configure phase was faster the change would be even more substantial.

Description of changes

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 23.11 Release Notes (or backporting 23.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

[dasJ]

Since you are already touching the package, could you pick this darwin patch as well so Hydra is more likely to succeed with the rebuild?

[trofi]

Since you are already touching the package, could you pick this darwin patch as well so Hydra is more likely to succeed with the rebuild?

Sounds good! Will take me a few hours.

[trofi]

Added gnugrep: disable gnulib tests on x86_64-darwin as well. It's a slightly different but equivalent patch that uses equivalent helpers:

--- a/pkgs/tools/text/gnugrep/default.nix
+++ b/pkgs/tools/text/gnugrep/default.nix
@@ -19,2 +19,4 @@ stdenv.mkDerivation {
-  # Some gnulib tests fail on Musl: https://github.com/NixOS/nixpkgs/pull/228714
-  postPatch = if stdenv.hostPlatform.isMusl then ''
+  # Some gnulib tests fail
+  # - on Musl: https://github.com/NixOS/nixpkgs/pull/228714
+  # - on x86_64-darwin: https://github.com/NixOS/nixpkgs/pull/228714#issuecomment-1576826330
+  postPatch = if stdenv.hostPlatform.isMusl || (stdenv.hostPlatform.isDarwin && stdenv.hostPlatform.isx86_64) then ''

Tested eval as:

$ nix repl . --argstr system x86_64-darwin
nix-repl> gnugrep.postPatch
"sed -i 's:gnulib-tests::g' Makefile.in\n"

Looks expected.

[reckenrode]

gnugrep no longer hangs with the Darwin patch, but I get a test failure (Rosetta 2, macOS 13.4).

FAIL: stack-overflow
====================

./stack-overflow: line 31: 23183 Abort trap: 6           grep -E -f in > out 2> err
./stack-overflow: line 31: 23195 Abort trap: 6           grep -E -f in > out 2> err
./stack-overflow: line 31: 23208 Abort trap: 6           grep -E -f in > out 2> err
./stack-overflow: line 31: 23220 Abort trap: 6           grep -E -f in > out 2> err
./stack-overflow: line 31: 23233 Abort trap: 6           grep -E -f in > out 2> err
./stack-overflow: line 31: 23245 Abort trap: 6           grep -E -f in > out 2> err
./stack-overflow: line 31: 23257 Abort trap: 6           grep -E -f in > out 2> err
./stack-overflow: line 31: 23270 Abort trap: 6           grep -E -f in > out 2> err
./stack-overflow: line 31: 23285 Abort trap: 6           grep -E -f in > out 2> err
./stack-overflow: line 31: 23300 Abort trap: 6           grep -E -f in > out 2> err
./stack-overflow: line 31: 23325 Abort trap: 6           grep -E -f in > out 2> err
stack-overflow: failed test: grep never printed "stack overflow"
FAIL stack-overflow (exit status: 1)

I modified the test script to cat the actual contents of err. It doesn’t look like this test plays very nicely with Rosetta 2. This is what it shows:

rosetta error: unexpectedly need to EmulateForward on a synchronous exception x86_rip=0x4303486096 arm_pc=0x4303949136 num_insts=6 inst_index=4 x86 instruction bytes: 0x6215344901283465301 0x17041981987679720769

[trofi]

Looks like an emulator deficiency. Is it generally able to run testsuites of most other programs?

[reckenrode]

Looks like an emulator deficiency. Is it generally able to run testsuites of most other programs?

Normally, yes.

This particular test creates a pathological file with a million left-parens then tries to grep it. The test wants grep to crash, but it doesn’t crash with the expected error message (grep: stack overflow), so the test fails.

If you perform the test manually (create the file, try to grep it), it does crash in the expected way. There’s something about the test environment it sets up that causes the Rosetta 2 error.

[trofi]

Aha, that makes sense. I wonder if gnugrep's failure something non-trivial (like initial environment size) or a sandbox configuration bug (if darwin implements sandbox) where we don't pass through enough into the build environment for emulator to just work.

[reckenrode]

Darwin support sandboxing, which normally works with Rosetta 2. However, while I normally have the sandbox enabled, it was disabled when I captured the output with the Rosetta 2 error.

[trofi]

Added gnugrep: disable tests x86_64-darwin to disable tests on x86_64-darwin as:

--- a/pkgs/tools/text/gnugrep/default.nix
+++ b/pkgs/tools/text/gnugrep/default.nix
@@ -33 +33,2 @@ stdenv.mkDerivation {
-  doCheck = !stdenv.isCygwin && !stdenv.isFreeBSD;
+  # x86_64-darwin: fails 'stack-overflow' tests on Rosetta 2 emulator
+  doCheck = !stdenv.isCygwin && !stdenv.isFreeBSD && !(stdenv.isDarwin && stdenv.hostPlatform.isx86_64);

[trofi]

Let's declare it good enough for staging.