perlPackages: Switch to SRI hashes, add hash support to bootstrap fetchurl, bump minimal nix version

[dasJ]

The future is now!

Description of changes

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 22.11 Release Notes (or backporting 22.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
  • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
• Fits CONTRIBUTING.md.

[dasJ]

Also cc @edolstra and @globin because of the fetchurl change. It seems reasonable but is it a good idea?
I verified that corepkgs fetchurl has a hash attribute from version 2.3 onwards

[edolstra]

Suggested change

	- nixpkgs now requires nix 2.3 or newer.
	- Nixpkgs now requires Nix 2.3 or newer.

Yes I'm being anal about capitalisation :-)

[dasJ]

Thanks, I didn't know that Nixpkgs and Nix are to be capitalized. Learned something along the way ;)

[FRidh]

Nix 2.3 is already several years old so bumping that should be fine IMO.

How are the hashes for the Perl packages obtained? There is a RFC about the usage of SRI hashes in Nixpkgs NixOS/rfcs#131. There was a discussion recently about the usage of those hashes in the Python packages set. On PyPI they are shown in a different base. Using SRI hashes means you cannot directly compare hashes and this was considered a disadvantage. What is your view on that in case of Perl packages?

[dasJ]

@FRidh there are no hashes shown on metacpan as far as I can tell, although they can be obtained using the API.
I have no personal opinion on this as I doubt there are people actively comparing the hashes between nixpkgs and metacpan (although @stigtsp and @zakame may have a different opinion). The gain in consistency is worth it in my opinion as there is no user interface that would show the hash in question. Somebody crawling the API can easily format the hash as well to get the correct format.

Also, the Perl package set has not been updated in a long time, but the tool that @stigtsp is devloping to do that in the future uses SRI hashes as well.

Edit: Sorry @stig, I pinged the wrong stig :(

[SuperSandro2000]

https://github.com/NixOS/nix/blob/master/src/libexpr/fetchurl.nix is not checking if both are set. We should assert that here.

[xworld21]

Is there a reason for allowing fetchurl without any hashes? I figure that to preserve the old behaviour, the code should also assert that at least one hash is set (and use null as default value, to distinguish from an intentionally empty value).

[dasJ]

Last time I tried ofborg failed evaluation because of this, although I'm not able to reproduce this locally.

[dasJ]

Looks like I was too tired yesterday and forgot how boolean logic works :/ Pushed the change

[xworld21]

I see that the other fetchurl has hash ? "", so I should drop my comment about null! All good for me.

[Artturin]

now it's not possible to set the hash to "" and TOFU

It works in fetchurl/default

nixpkgs/pkgs/build-support/fetchurl/default.nix

Lines 123 to 134 in e04579e

	hash_ =
	# Many other combinations don't make sense, but this is the most common one:
	if hash != "" && sha256 != "" then throw "multiple hashes passed to fetchurl" else
	if hash != "" then { outputHashAlgo = null; outputHash = hash; }
	else if md5 != "" then throw "fetchurl does not support md5 anymore, please use sha256 or sha512"
	else if (outputHash != "" && outputHashAlgo != "") then { inherit outputHashAlgo outputHash; }
	else if sha512 != "" then { outputHashAlgo = "sha512"; outputHash = sha512; }
	else if sha256 != "" then { outputHashAlgo = "sha256"; outputHash = sha256; }
	else if sha1 != "" then { outputHashAlgo = "sha1"; outputHash = sha1; }
	else if cacert != null then { outputHashAlgo = "sha256"; outputHash = ""; }
	else throw "fetchurl requires a hash for fixed-output derivation: ${lib.concatStringsSep ", " urls_}";

[stigtsp]

@FRidh there are no hashes shown on metacpan as far as I can tell, although they can be obtained using the API.
[..]

I agree that the format of the hashes probably don't matter that much, since they decode to the same bytes.

$ nix hash to-sri --type sha256 $(curl -s https://fastapi.metacpan.org/v1/release/RGARCIA/Perl-Destruct-Level-0.02 | jq -r .release.checksum_sha256)
sha256-QLSsCykrYM47h956o5vC+yWhnRDlyfaYZpYchLP20Ts=

Since some issues have been highlighted recently with verifying CHECKSUMS files generated by CPAN/PAUSE, using the MetaCPAN API seems to be the best source of these hashes.

Thanks again @dasJ for working on perlPackages ❤️

[xworld21]

I see that the other fetchurl has hash ? "", so I should drop my comment about null! All good for me.

[stigtsp]

(Huge) diff LGTM

Issue with DB_File

[stigtsp]

preConfigure in perlPackages.DBFile was inadvertently modified (can't comment inline since the diff is too huge)

@@ -6826,13 +6826,13 @@ let
 
     src = fetchurl {
       url = "mirror://cpan/authors/id/P/PM/PMQS/DB_File-1.855.tar.gz";
-      sha256 = "0q599h7g4jkzks5dxf1zifx9k7l9vif26r2dlgkzxkg6bfif5zyr";
+      hash = "sha256-2f/iolvmzf7no01kI1zciZ6Zuos/uN6Knn9K8g5MqWA=";
     };
 
     preConfigure = ''
       cat > config.in <<EOF
       PREFIX = size_t
-      HASH = u_int32_t
+      hash = u_int32_t
       LIB = ${pkgs.db.out}/lib
       INCLUDE = ${pkgs.db.dev}/include
       EOF

[stigtsp]

The future LGTM!

[stig]

Edit: Sorry @stig, I pinged the wrong stig :(

Don't worry about it! It's not the first time someone does that 😄