cgroup driver setup fails: failed to run Kubelet: misconfiguration: kubelet cgroup driver: \"cgroupfs\" is different from docker cgroup driver: \"systemd\"

[ysicing]

Environmental Info:
k3s version v1.23.7+k3s1 (ec61c66)
go version go1.17.5

Node(s) CPU architecture, OS, and Version:

Linux k3s01 5.17.0-3-amd64 #1 SMP PREEMPT Debian 5.17.11-1 (2022-05-26) x86_64 GNU/Linux

Cluster Configuration:

1 server

Describe the bug:

K3s now uses the systemd cgroup driver instead of cgroupfs when running under systemd 244 or later. #5462

Steps To Reproduce:

• Installed K3s:

# docker /etc/docker/daemon.json

{
  "exec-opts": [
    "native.cgroupdriver=systemd"
  ],
  "bip": "169.254.123.1/24",
  "storage-driver": "overlay2"
}

# k3s
k3s server --docker

Expected behavior:

K3S should start.

Actual behavior:

"Failed to run kubelet" err="failed to run Kubelet: misconfiguration: kubelet cgroup driver: \"cgroupfs\" is different from docker cgroup driver: \"systemd\""

Additional context / logs:

Backporting

• Needs backporting to older releases

[brandond]

If you manually change the cgroup driver for docker to systemd, you should change it for the kubelet too, with --kubelet-arg=cgroup-driver=systemd. K3s only manages the driver when using the packaged containerd.

You should be aware that dockershim is gone as of 1.24; k3s will no longer support the --docker flag.

[ysicing]

thank you. I thought k3s would config it automatically