nixos/unifi controller: mongod crashes immediately (Raspberry Pi; aarch64)

[alexvorobiev]

Describe the bug
Unifi controller service does not work because mongodb coredumps immediately.

To Reproduce

services.unifi = {
    enable = true;
    unifiPackage = pkgs.unifiStable;
    openPorts = true;
    jrePackage = pkgs.jre8_headless;
  };

Additional context

[root@rpi:~]# systemctl status unifi
● unifi.service - UniFi controller daemon
   Loaded: loaded (/nix/store/fy5spfhm6rsfqizi08fqihknxza816m1-unit-unifi.service/unifi.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2019-12-06 22:13:22 CST; 3min 0s ago
  Process: 771 ExecStartPre=/nix/store/0mv4hzai2vk75h2528k34ms5i7pxhzly-unit-script-unifi-pre-start (code=exited, status=0/SUCCESS)
 Main PID: 779 (java)
       IP: 47.9K in, 14.6K out
    Tasks: 22
   Memory: 272.7M
      CPU: 1min 7.886s
   CGroup: /system.slice/unifi.service
           ├─ 779 java -jar /var/lib/unifi/lib/ace.jar start
           └─1099 bin/mongod --version

Dec 06 22:16:01 rpi systemd-coredump[1030]: Process 1028 (mongod) of user 183 dumped core.
Dec 06 22:16:04 rpi systemd-coredump[1033]: Process 1031 (mongod) of user 183 dumped core.
Dec 06 22:16:06 rpi systemd-coredump[1048]: Process 1046 (mongod) of user 183 dumped core.
Dec 06 22:16:09 rpi systemd-coredump[1051]: Process 1049 (mongod) of user 183 dumped core.
Dec 06 22:16:12 rpi systemd-coredump[1066]: Process 1064 (mongod) of user 183 dumped core.
Dec 06 22:16:14 rpi systemd-coredump[1069]: Process 1067 (mongod) of user 183 dumped core.
Dec 06 22:16:19 rpi systemd-coredump[1085]: Process 1083 (mongod) of user 183 dumped core.
Dec 06 22:16:20 rpi systemd-coredump[1088]: Process 1086 (mongod) of user 183 dumped core.
Dec 06 22:16:24 rpi systemd-coredump[1101]: Process 1099 (mongod) of user 183 dumped core.
Dec 06 22:16:26 rpi systemd-coredump[1106]: Process 1104 (mongod) of user 183 dumped core.

There is nothing useful in the log other than a series of

[2019-12-06T21:30:53,998] <db-server> INFO  db     - DbServer stopped

mongod.log is empty.

I found working mongodb in an older nixpkgs (one of the prior generations I haven't GC'd yet) so now I have this in my configuration.nix (using the approach from this post):

 services.unifi = {
    enable = true;
    unifiPackage = pkgs.unifiStable;
    openPorts = true;

    mongodbPackage = let
      channelRelease = "nixos-19.09pre190687.3f4144c30a6";  # last known working mongo
      channelName = "unstable";
      url = "https://releases.nixos.org/nixos/${channelName}/${channelRelease}/nixexprs.tar.xz";
      sha256 = "040f16afph387s0a4cc476q3j0z8ik2p5bjyg9w2kkahss1d0pzm";

      pinnedNixpkgsFile = builtins.fetchTarball {
        inherit url sha256;
      };

      pinnedNixpkgs = import pinnedNixpkgsFile {};
    in pinnedNixpkgs.mongodb;

    jrePackage = pkgs.jre8_headless;
  };

Metadata
Please run nix run nixpkgs.nix-info -c nix-info -m and paste the result.

 - system: `"aarch64-linux"`
 - host os: `Linux 5.4.1, NixOS, 20.03pre204199.3140fa89c51 (Markhor)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.3.1`
 - channels(root): `"nixos-20.03pre204199.3140fa89c51"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute: 
  - unifi-controller
  - mongodb
# a list of nixos modules affected by the problem
module: unifi

[ghost]

I just ran into the same issue. Thanks for providing your workaround, it saved me quite some time :)

[colemickens]

Wow, another +1 from me, thanks for sharing the workaround!

[stale]

Thank you for your contributions.

This has been automatically marked as stale because it has had no activity for 180 days.

If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.

Here are suggestions that might help resolve this more quickly:

1. Search for maintainers and people that previously touched the related code and @ mention them in a comment.
2. Ask on the NixOS Discourse.
3. Ask on the #nixos channel on irc.freenode.net.

[ghost]

Still important to me

[colemickens]

I just refactored my config and I think this might be fixed. I tested $(nix-build ...nixos-unstable... -A mongodb)/bin/mongod and it started. I removed this workaround and the controller still starts up.

As proof, in the workaround, mongodb is at version 3.4.10: https://github.com/NixOS/nixpkgs/blob/3f4144c30a6351dd79b177328ec4dea03e2ce45f/pkgs/servers/nosql/mongodb/default.nix

And on my system, the controller is running and from systemctl cat var-lib-unifi-bin.mount:

...
What=/nix/store/b614d94didlhk1x6lfy5aiswjy1hh0mp-mongodb-3.4.24/bin
Where=/var/lib/unifi/bin
...

[alexvorobiev]

@colemickens Did you tried booting with this configuration? I have just tried updating the unstable channel on my rpi and it won't even boot. It freezes on "Starting kernel ...". Rolling back to the previous configuration makes it boot as usual.

[colemickens]

I just rebooted it, it is fine. This is my current unifi file: https://github.com/colemickens/nixcfg/blob/fdf283369f97e8462fbc38ecfbbc967a60af1bdb/mixins/unifi.nix

and the generation is : /nix/store/hial9051vxkgxvxynifrf6zkhn7v6r77-nixos-system-raspberry-20.09.20200719.5717d9d which is https://github.com/nixos/nixpkgs/tree/5717d9d, a very recent commit.

[colemickens]

(I'm running a slightly different module than the one in the nixpkgs tree and I know it had a change in the last month though, not sure if that's related to what you're seeing.)

[alexvorobiev]

It looks like I am a victim of #82455 (rpi boots but the screen freezes). There is now also #97064. I will explore pinning the kernel using boot.kernelPackages = pkgs.linuxPackages_X_XX.

[stale]

I marked this as stale due to inactivity. → More info

[Thra11]

I had this issue with mongodb on aarch64 (not raspberry pi specifically) and was using the pinned version for a while (thanks!). I've now removed the pin and it seems to be working ok without (I'm currently on unstable 9a3f486), so I assume this issue can be closed?

[kalbasit]

Mine is working well too.

[simonchatts]

For completeness, #91818 was the fix.