-
-
Notifications
You must be signed in to change notification settings - Fork 13.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Preview: deterministic build #2281
Preview: deterministic build #2281
Conversation
Known issues:
|
I think nix uses the equivalent of |
Wonderful. @alexanderkjeldaas that means most of other PRs can be closed? |
@vcunat @alexanderkjeldaas Can we get some of these things merged on the pending stdenv branch? I know that we haven't solved the GCC question re: PGO, so we'll have to leave that commit out, but a lot of these changes are not very intrusive, and merging them would reduce burdens later and get us much closer to a deterministic build. |
Ah, I completely forgot this series of work. The currently staged stdenv has been quite tested by me, so I would merge it about the current state (after Hydra verifies gcc on Darwin). I'll look at this afterwards, as I wanted to do another iteration of stdenv stuff (there were some others I missed this time). |
@vcunat I'll just leave it as-is then I guess. On Mon, Jun 9, 2014 at 7:40 PM, Vladimír Čunát notifications@github.com
|
@alexanderkjeldaas I'm going to begin merging some of this work into HEAD soon. I'm probably not going to merge everything in one go, so feel free to rebase this when you get a chance. I'll update with what I've pushed upstream. |
I've rebased On Sun, Jun 29, 2014 at 3:47 AM, Austin Seipp notifications@github.com
|
I've added a minor fix for python 2.7.7 that I forgot to cherry-pick from my internal branch. |
So, what is the status of cherry-picking? Github doesn't easily show this, unfortunately… Obviously, this will never get directly in master (only in staging) and it gets chery-picked in small pieces. I actually support reproducibility, although some people seem to like PGO too much… |
PGO is probably the only questionable thing here, IIRC. I'm planning to really review and test this within the next 10 days. |
Remove datetime from nscd.
This includes two changes: 1) Fix a bug where the bootstrap-tools is always used instead of binutils 2) Enable strip --enable-deterministic-archives as soon as a new binutils is available.
1) Make the core python libraries deterministic. 2) Make the python libraries created by glib deterministic.
Note that 3157dbe is probably no longer required due to tytso/e2fsprogs@a2143b5 |
I've experimented with patching |
Hm, I guess |
I've been going through most of these. A brief summary so far: I think the following are obsolete
The following need more work
I've not looked at libgpg-error, busybox, python, perl, or the gcc stuff, nor the general libfaketime support. |
Regarding the manual, https://wiki.debian.org/ReproducibleBuilds/ExperimentalToolchain#libxslt indicates that the issue of random ids is fixed/worked on upstream, so perhaps we want to just skip that for now. |
something that can help to stress-test determinism, http://manpages.ubuntu.com/manpages/xenial/man1/disorderfs.1.html this is a fuse filesystem that randomizes the order of files in a directory, so you cant accidentally rely on the fs doing that most of the time |
Some of the perl stuff was lost when perl16 was removed, but otherwise it seems like @vcunat (or whomever) picked up most of the specific package fixes. I think we're left with gcc, faketime in stdenv, stdenv numbering, and the fake date command thing, all of which seem like they could be profitably dealt with on their own. |
IIRC I had tried hard to pick whatever I could verify/make clearly advantegous. For some issues (like PGO) I didn't succeed in a reasonable amount of time, so I left those behind. |
FYI this PR is being linked to from https://reproducible-builds.org/who/. I volunteer to ping whoever's necessary to get an update posted (don't have the knowledge for more, unfortunately). |
(triage) My reading of the comments in this thread is that most of the changes either have been merged or are no longer needed. The remaining changes that would require being split out to separate PRs would be:
Does that sound correct to those actually involved? |
Note: the GCC stuff (force-setting |
The fake date command is no longer necessary, where faketime by itself is sufficient, so I suppose the remaining points are 1. numbering the names of stdenv, and 2. using faketime in stdenv. |
Are there any updates on this pull request, please? |
This is not actionable in its current state. Please open issues with references to the changes within this PR if needed. |
There is a GitHub project for this topic: https://github.com/NixOS/nixpkgs/projects/10 @worldofpeace can you add this and every related issue (even closed) to it? Then https://reproducible-builds.org/who/ can link to that. The current status is visible on https://r13y.com/ (at least for nixos-unstable's iso_minimal). |
Perhaps it might make more sense to have one ticket for each (important) irreproducible package? (or "unreproducible"? whatever) At least for those that someone started to investigate and didn't immediately manage to fix it completely. |
Sure, it makes sense to have issues in the tracker for this purpose. Though I'm not sure if I'll have the time to distill them from this thread. @vcunat that is an important point for those who did take time to investigate that |
This is a set of changes that makes the system_tarball_pc derivation deterministic.
Stdenv bootstrap:
Stdenv builder:
Gcc:
Various changes: