Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Push source into nix/ subdir to avoid annoying extra rebuilds #5589

Closed
wants to merge 4 commits into from

Conversation

Ericson2314
Copy link
Member

Currently, flake.nix, docker.nix, etc. "pollute" the most of the nix build's derivations, even though these files are not need at build time. This makes end-to-end testing of builds rather annoying, too many edits are "mass rebuilds".

Rather than do lots of complicated filtering, we just moved Nix itself to be in a sub-directory, just like perl/ for the perl bindings. This means that we can take whole subdirs, which, after the usual git-based filtering, means we get what we want.

The churn of so many files being moved might raise some eyebrows, but I this this is the right approach. Git handles "pure" renames quite well (as opposed to file splits), so I don't expect this to negatively impact other developers.

Even more importantly, we want the Nix repo to be a good example for (potential) users of Nix of best practices. I think it's important people remember not thinking fancy filter is the best/only way to do things, and I also certainly don't want them to think that extra rebuilds are the inevitable cost of purity. This avoids making both negative impressions.

The one caveat is that without making officialRelease = true, versionSuffix will end up tainting everything with the commit hash anyways. I have just been setting officialRelease = true and remembering not to commit it, which works but we can do better. I am not sure what the right thing to do is, and am certainly open to suggestions.

Mostly fixes #5574

@Ericson2314 Ericson2314 changed the title Partition source to avoid annoying extra rebuilds Push source into Nix subdir to avoid annoying extra rebuilds Nov 18, 2021
@Ericson2314 Ericson2314 changed the title Push source into Nix subdir to avoid annoying extra rebuilds Push source into nix/ subdir to avoid annoying extra rebuilds Nov 18, 2021
@edolstra
Copy link
Member

I don't think this is a good approach. Being forced to move almost the entire source tree into a subdirectory just to work around a Nix issue is not best practice and not something most projects would want to do. I mean, surely we don't want to tell users that adding a flake.nix requires reorganizing their entire repository.

@Ericson2314
Copy link
Member Author

Ericson2314 commented Nov 18, 2021

@edolstra this has nothing to do with flakes I now see. Proper use of Nix with or without flakes does require separation components into subdirectories to avoid silly rebuilds unless we want to write lots of fancy fitlerSource, and that is good practice anyways even without Nix to make logical boundaries clear.

Remember, this is not a single-package repo, for nix itself and the perl bindings are separate packages.

ElvishJerricco and others added 4 commits February 21, 2023 11:23
This avoids rebuilding all of Nix whenever you change a Nix expression
in flake.nix
The configure script can sanity check it has the right source directory
with a different file instead. `AC_CONFIG_SRCDIR` before `AC_INIT` is
fine.
Avoids rebuilds, but also good to limit what stuff we leak from `./nix` in
general.
@Ericson2314
Copy link
Member Author

Ericson2314 commented Feb 21, 2023

On pause until a decision is reached in #7876.

@Ericson2314 Ericson2314 deleted the self-src-swap branch October 20, 2023 13:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Nix's build system should filterSource or similar
3 participants