Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run unit tests with clean config #10562

Merged
merged 4 commits into from
Apr 21, 2024

Conversation

roberth
Copy link
Member

@roberth roberth commented Apr 19, 2024

Motivation

@edolstra found an error running the tests in the dev shell.

It turns out the normal initialization still ran, and loaded configuration that caused the error.

This PR disables the loading of external configuration during unit test execution, and makes it possible for test suites for C API based bindings to do the same.

Context

The error was:

tests/unit/libutil-support/tests/nix_api_util.hh:34: Failure
Failed
nix_err_code(ctx) != NIX_OK, message: error: creating cgroup '/sys/fs/cgroup/user.slice/user-1000.slice/session-7.scope/nix-build-pid-2087076-2': Permission denied
tests/unit/libexpr/nix_api_expr.cc:162: Failure

This error was only recently possible, as #10412 was the first PR to run small builds as part of the unit tests.
Building in unit tests should perhaps be kept to a minimum, but I have to note:

  • single derivations are pretty fast to build
  • we need to be able to build in various environment for the functional tests anyway
  • this was the only way to test that part of the C API
  • mocking only works if you have exceptionally good interfaces and the mocking code is reviewed over and over, if it works at all

Priorities and Process

Add 👍 to pull requests you find important.

The Nix maintainer team uses a GitHub project board to schedule and track reviews.

@github-actions github-actions bot added the c api Nix as a C library with a stable interface label Apr 19, 2024
namespace nix {

int preInitForTests(int argc, char ** argv)
{
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unchanged code from here down.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The libstore test suite ironically did not exhibit the problem, because it was the C API tests that triggered the problem, which are mostly libexpr focused. (nix_string_realise)

Unit tests that exercise build code should be possible to add though, so I've applied the same solution here.

@edolstra
Copy link
Member

Wouldn't it be easier to do what we do in tests/functional/common/vars-and-functions.sh, i.e. set $HOME etc. to known values? That way it wouldn't need any code changes.

@roberth
Copy link
Member Author

roberth commented Apr 19, 2024

I think it's cleaner to not even look in those places.

Ideally we wouldn't even have such globals, and then we can trivially clean up the new code I wrote here.

@roberth
Copy link
Member Author

roberth commented Apr 19, 2024

Doing it in C++ also helps with portability.

@Ericson2314
Copy link
Member

There are global variables in libutil I am afraid too (the experimental feature settings).

Would it be possible do this less imperatively in the C++, e.g. adding arguments to the C++ init functions? (OK if the C interface is still stateful since compat is more of an issue there.)

@github-actions github-actions bot added the with-tests Issues related to testing. PRs with tests have some priority label Apr 19, 2024
@roberth
Copy link
Member Author

roberth commented Apr 19, 2024

Good call. A new flag is a much simpler solution, and we don't need a particularly extension pattern at this time (or ever, hopefully).
On the C side, where we don't have overloading, it's just a new function.

@roberth
Copy link
Member Author

roberth commented Apr 19, 2024

Part of why the diff is smaller now is that I'll do separately the part where libstore tests become capable of building.
That's not needed yet, but good prep nonetheless for the build improvements that may be coming, so I'll do that in a followup.

@@ -9,6 +9,7 @@
#include "path.hh"
#include "derived-path.hh"
#include "exit.hh"
#include "globals.hh"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still needed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It wasn't!

Copy link
Member

@Ericson2314 Ericson2314 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Except for my little note

This reverts commit 62feb5c.

It runs as part of the functional tests, which control the environment,
solving some of the problems a default config has when run in the
sandbox.
@roberth roberth merged commit d871e7c into NixOS:master Apr 21, 2024
9 checks passed
roberth added a commit to edolstra/nix that referenced this pull request Apr 21, 2024
Fixed in NixOS#10562

Building in unit tests should perhaps be kept to a minimum, but I have to note:

* single derivations are pretty fast to build
* we need to be able to build in various environment for the functional tests anyway
* this was the only way to test that part of the C API
* mocking only works if you have exceptionally good interfaces and the mocking code is reviewed over and over, if it works at all

For what it's worth, I think we can have an "exceptionally good interface",
in the form of NixOS#10579
Until that's been worked out, these cheap builds are fine to run in unit tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug c api Nix as a C library with a stable interface tests with-tests Issues related to testing. PRs with tests have some priority
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants