I'm trying to get K0s working in LXC (LXD) containers on Ubuntu 20.04 using ZFS and I've managed to get past the apparmor challenges (for now at least) but stuck on overlayfs issues during the install phase.
kernel: [1617529.987422] overlayfs: filesystem on '/var/lib/k0s/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/6371/fs' not supported as upperdir
Has anyone hit on this issues before and has a solution/workaround?
Regards, Matt
containerd needs to be told to use the zfs snapshotter:
This approach will probably require the LXD container to run privileged & require the zfs modules passed through to the container.
With k0s it is also useful during the configuration above so containerd is found to add to your bash profile:
export PATH=$PATH:/var/lib/k0s/binAlternative Approach (unprivileged LXD)
Adding an ext4 zvol to the container for containerd will probably be less problematic & is better from a security perspective as the container can run unprivileged. This approach works with k3s & /var/lib/rancher/k3s/agent/containerd running on LXD with zfs:
/var/lib/kubelet seems to mount itself under zfs ok on k3s version v1.22.7+k3s1
Using rootless overlayfs for the snapshotter works now on Ubuntu 22.04 thanks to this patch. It uses less space (639MB versus 1.9GB) & gives better performance than the native snapshotter.
