GHA: add basic eval checks #129427

domenkozar · 2021-07-06T11:06:52Z

Until ofborg supports aarch64-darwin we can just run the tarball checks on GHA to make sure evaluation errors don't slip by.

zowoq · 2021-07-06T11:25:15Z

.github/workflows/tarball.yml

+
+jobs:
+  tests:
+    runs-on: ubuntu-latest


Suggested change

runs-on: ubuntu-latest

runs-on: ubuntu-latest

if: github.repository_owner == 'NixOS'

I'm not sure that's really needed, I'd like my forks to check for evaluation errors before I make a PR.

Add a comment then as this will be the only one without the restriction.

vcunat · 2021-07-06T11:30:36Z

.github/workflows/tarball.yml

+    steps:
+    - uses: actions/checkout@v2
+    - uses: cachix/install-nix-action@v13
+    - run: nix-build pkgs/top-level/release.nix -A tarball --arg supportedSystems '[ "x86_64-linux" "x86_64-darwin" "aarch64-linux" "aarch64-darwin" ]'


Nit: maybe at least move it to the first place? (Otherwise it takes quite some time to get to the error specific for aarch64-darwin, and we don't check it anywhere else so far.)

OOMs with all anyway:
https://github.com/NixOS/nixpkgs/pull/129427/checks?check_run_id=2998217938 https://github.com/NixOS/nixpkgs/pull/129427/checks?check_run_id=2998218312

I've pushed a fix that make swap partition 8GB. 🤞

I just checked the runtime of the checks and so far they are not done after 67 minutes which is a bit on the long side.

domenkozar · 2021-07-06T16:43:21Z

I've removed (temporarily) the two checks that take each 15GB and 19GB of memory.

Now the job takes 10min.

How about we add a boolean flag to the tarball build like disableMemoryIntensiveChecks that remove these two checks? These two are then run by ofborg anyway (and they don't depend on the system).

vcunat · 2021-07-06T17:18:57Z

I had previously thought about splitting it into multiple builds (multiple *.drv). Also to get possibility of distribution/parallelization.

domenkozar · 2021-07-07T12:27:29Z

I've split the basic checks in a separate derivation and made GHA only build that.

The checks only trigger on master/release-* branches.

Ideally we'd also run the two full evals, but they each take 16GB and 19GB of memory which takes quite a while due to a large swap usage.

Mic92 · 2021-07-08T06:19:26Z

.github/workflows/basic-eval.yml

+    - uses: actions/checkout@v2
+    - uses: cachix/install-nix-action@v13
+    # explicit list of supportedSystems is needed until aarch64-darwin becomes part of the trunk jobset
+    - run: nix-build pkgs/top-level/release.nix -A tarball.nixpkgs-basic-release-checks --arg supportedSystems '[ "aarch64-darwin" "aarch64-linux" "x86_64-linux" "x86_64-darwin"  ]'


Would gives us a build matrix a 4x speedup here?

We also need to make sure people to not try to run this command on an older nixpkgs checkout or they are greeted with this error message: error: attribute 'nixpkgs-basic-release-checks' in selection path 'tarball.nixpkgs-basic-release-checks' not found

Could that happen? This .yml is in-repo, so I don't expect it would naturally act on older commits.

🤔 I think you are right. People wouldn't run into this when they use an older checkout.

Would gives us a build matrix a 4x speedup here?

Not entirely since there's overhead of the job doing other checks as well.

I'm a bit hestiant to do that not to hit the jobs limit on our repo.

What is the job limit? Is there any sharing between different architectures apart from downloading & installing nix?

20 concurrent

pkgs/top-level/make-tarball.nix

SuperSandro2000 · 2021-07-08T09:16:30Z

.github/workflows/basic-eval.yml

+jobs:
+  tests:
+    runs-on: ubuntu-latest
+    # we don't limit this action to only NixOS repo since the checks are cheap and useful developer feedback


I don't agree with this one. Normally when the tarball does not build on hydra the error messages are always very cryptic or do not display any helpful information at all. So far I was not able to fix any of those build failures on hydra when they blocked a channel advancement and I think my nix knowledge is above average and I should be able to fix that or at least diagnose the problem.

That's an orthogonal problem? This job only gives you feedback that it will block the channel if merged. So the issue has to be addressed before the PR goes in.

Hmmm.. Maybe we should just try it and see how it goes.

Co-authored-by: Sandro <sandro.jaeckel@gmail.com>

domenkozar · 2021-07-08T14:57:45Z

Please take another look :)

domenkozar · 2021-07-13T07:56:18Z

I'm going to merge this in a day unless there are any objections.

Mic92 · 2021-07-17T05:42:28Z

On staging it should probably not use nix from nixpkgs but from a hydra cached version: https://github.com/NixOS/nixpkgs/pull/130425/checks?check_run_id=3091788566

Mhm sees is probably already the case. Is there a way to evaluate what the tarball evaluates without doing a stdenv rebuild?

domenkozar · 2021-07-17T06:13:33Z

It's supposed to trigger only for master/release branches

Mic92 · 2021-07-17T08:34:23Z

Mhm. Okay. Maybe my own fault, because the branch was target to master first.

samueldr · 2021-09-27T20:05:12Z

pkgs/top-level/nixpkgs-basic-release-checks.nix

+    echo 'abort "Illegal use of <nixpkgs> in Nixpkgs."' > $TMPDIR/barf.nix
+
+    # Make sure that Nixpkgs does not use <nixpkgs>.
+    badFiles=$(find $src/pkgs -type f -name '*.nix' -print | xargs grep -l '^[^#]*<nixpkgs\/' || true)


Is this expected that this greps through nix files, while hitting comments?

domenkozar requested review from Mic92, zowoq and a team as code owners July 6, 2021 11:06

github-actions bot added the 6.topic: policy discussion label Jul 6, 2021

domenkozar mentioned this pull request Jul 6, 2021

Support for Apple Silicon (aarch64-darwin) #95903

Closed

domenkozar force-pushed the gha-tarball branch from 5fa2618 to 4457780 Compare July 6, 2021 11:14

domenkozar mentioned this pull request Jul 6, 2021

darwin.apple_sdk.frameworks.CoreSymbolication: expose under SDK 11.0 #129292

Merged

11 tasks

ofborg bot added 10.rebuild-darwin: 0 10.rebuild-linux: 0 labels Jul 6, 2021

zowoq reviewed Jul 6, 2021

View reviewed changes

vcunat reviewed Jul 6, 2021

View reviewed changes

domenkozar force-pushed the gha-tarball branch 3 times, most recently from 588e182 to b028552 Compare July 6, 2021 13:22

domenkozar force-pushed the gha-tarball branch from 6f23d96 to b028552 Compare July 7, 2021 12:17

domenkozar changed the title ~~GHA: add tarball checks~~ GHA: add basic eval checks Jul 7, 2021

GHA: add basic eval checks

7904fff

domenkozar force-pushed the gha-tarball branch from b028552 to 7904fff Compare July 7, 2021 12:27

Mic92 reviewed Jul 8, 2021

View reviewed changes

SuperSandro2000 reviewed Jul 8, 2021

View reviewed changes

pkgs/top-level/make-tarball.nix Outdated Show resolved Hide resolved

SuperSandro2000 reviewed Jul 8, 2021

View reviewed changes

Update pkgs/top-level/make-tarball.nix

81a35d6

Co-authored-by: Sandro <sandro.jaeckel@gmail.com>

domenkozar merged commit 9ea790e into master Jul 14, 2021

vcunat deleted the gha-tarball branch July 14, 2021 07:37

samueldr reviewed Sep 27, 2021

View reviewed changes

samueldr mentioned this pull request Sep 27, 2021

Add Plasma Mobile Gear 21.05 #125156

Merged

10 tasks

zowoq mentioned this pull request Mar 31, 2022

.github/workflows/basic-eval.yml: try to parallelize #166599

Closed

13 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GHA: add basic eval checks #129427

GHA: add basic eval checks #129427

domenkozar commented Jul 6, 2021

zowoq Jul 6, 2021

domenkozar Jul 6, 2021

zowoq Jul 6, 2021

vcunat Jul 6, 2021

SuperSandro2000 Jul 6, 2021

domenkozar Jul 6, 2021

SuperSandro2000 Jul 6, 2021

domenkozar commented Jul 6, 2021 •

edited

Loading

vcunat commented Jul 6, 2021 •

edited

Loading

domenkozar commented Jul 7, 2021 •

edited

Loading

Mic92 Jul 8, 2021

SuperSandro2000 Jul 8, 2021

vcunat Jul 8, 2021

SuperSandro2000 Jul 8, 2021

domenkozar Jul 8, 2021

Mic92 Jul 10, 2021 •

edited

Loading

domenkozar Jul 10, 2021

SuperSandro2000 Jul 8, 2021

domenkozar Jul 8, 2021

SuperSandro2000 Jul 8, 2021

domenkozar commented Jul 8, 2021

domenkozar commented Jul 13, 2021

Mic92 commented Jul 17, 2021 •

edited

Loading

domenkozar commented Jul 17, 2021

Mic92 commented Jul 17, 2021

samueldr Sep 27, 2021

	runs-on: ubuntu-latest
	runs-on: ubuntu-latest
	if: github.repository_owner == 'NixOS'

GHA: add basic eval checks #129427

GHA: add basic eval checks #129427

Conversation

domenkozar commented Jul 6, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

domenkozar commented Jul 6, 2021 • edited Loading

vcunat commented Jul 6, 2021 • edited Loading

domenkozar commented Jul 7, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Mic92 Jul 10, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

domenkozar commented Jul 8, 2021

domenkozar commented Jul 13, 2021

Mic92 commented Jul 17, 2021 • edited Loading

domenkozar commented Jul 17, 2021

Mic92 commented Jul 17, 2021

Choose a reason for hiding this comment

domenkozar commented Jul 6, 2021 •

edited

Loading

vcunat commented Jul 6, 2021 •

edited

Loading

domenkozar commented Jul 7, 2021 •

edited

Loading

Mic92 Jul 10, 2021 •

edited

Loading

Mic92 commented Jul 17, 2021 •

edited

Loading