zfs fails to mount /home from another zpool

[ghostbuster91]

Hi,

Whether this is an issue with disko or just more of a zfs+systemd general issue is beyond me. I am too much of a beginner with zfs at this point. Just looking for help mostly!

I have two disks in my laptop hence I figured out that I would create two separate zpools where one would have /nix, /var, / and the other one would have just /home. I don't want to but them into a single zpool because then when one fails everything breaks.

Here is the configuration I created for disko:

{ disks ? [ "/dev/vdb" "/dev/vdc" ], ... }: {
  disko.devices = {
    disk = {
      sda = {
        type = "disk";
        device = builtins.elemAt disks 0;
        content = {
          type = "table";
          format = "gpt";
          partitions = [
            {
              name = "ESP";
              start = "1MiB";
              end = "256MiB";
              fs-type = "fat32";
              bootable = true;
              content = {
                type = "filesystem";
                format = "vfat";
                mountpoint = "/boot";
              };
            }
            {
              name = "zfs";
              start = "256MiB";
              end = "100%";
              content = {
                type = "zfs";
                pool = "rpool1";
              };
            }
          ];
        };
      };
      sdb = {
        type = "disk";
        device = builtins.elemAt disks 1;
        content = {
          type = "table";
          format = "gpt";
          partitions = [
            {
              name = "zfs";
              start = "256MiB";
              end = "100%";
              content = {
                type = "zfs";
                pool = "rpool2";
              };
            }
          ];
        };
      };
    };
    zpool = {
      rpool1 = {
        type = "zpool";

        rootFsOptions = {
          compression = "lz4";
          "com.sun:auto-snapshot" = "false";
          canmount = "off";
        };
        options = {
          ashift = "12";
          autotrim = "on";
        };
        datasets = {
          zroot = {
            type = "zfs_fs";
            mountpoint = "/";
            postCreateHook = "zfs snapshot rpool1/zroot@blank";
          };
          znix = {
            type = "zfs_fs";
            mountpoint = "/nix";
            options = {
              atime = "off";
            };
          };
          zvar = {
            type = "zfs_fs";
            mountpoint = "/var";
            options = {
              acltype = "posixacl";
              xattr = "sa";
            };
          };
        };
      };
      rpool2 = {
        type = "zpool";

        rootFsOptions = {
          compression = "lz4";
          "com.sun:auto-snapshot" = "false";
          canmount = "off";
        };
        options = {
          ashift = "12";
          autotrim = "on";
        };
        datasets = {
          zhome = {
            type = "zfs_fs";
            mountpoint = "/home";
            postCreateHook = "zfs snapshot rpool2/zhome@blank";
            options = {
              "com.sun:auto-snapshot" = "true";
            };
          };
        };
      };
    };
  };
}

Problem

For some reason zfs fails to mount /home from rpool2 and enters emergency mode.

Logs from journalctl:

while in emergency mode df -h reports /rpool2/zhome to be mounted under /rpool2/zhome 🤔

but the /etc/fstab seems to be configured correctly:

Expected behavior

I would expect /home to be mounted correctly without any issues.

Additional info

After installing nixos I did:

# umount /mnt
# zpool export -fa

Finally, even though the boot goes to emergency mode due to the failing
mounting, simply continuing will then successfully mount the /home mount
without any further issues in use (df -h confirms that). So this is a somewhat minor issue.

I also read #114 as it seemed somehow similar.

Can it be that the native zfs mounting feature is trying to mount both zpools concurrently and because of that it is running into that mountpoint is busy error?

I guess it would be good to see if turning off zfs automount and switching to systemd (as mentioned in NixOS/nixpkgs#212762) would solve the problem, but I am not sure how to do that. (It seems that it could be the case https://discourse.nixos.org/t/my-zfs-volumes-fail-to-mount-at-boot/5445/6)

Should I just render the configuration as stated in #114 (comment) then modify it enabling legacy mountpoints and wire into configuration.nix instead of disko module?

[ghostbuster91]

After I switched to systemd for mounting it worked like a charm.

I did it by specifying options.mountpoint="legacy" for every dataset.

I had one minor issue with not being able to unmount zfs mountpoints after nixos installation presumably due to the reason as described in #211