0

For the automatic testing of my code I would like to add a microk8s in my docker-compose setup for testing. Hence, I'd like to install microk8s in docker environment

I got snap running in the docker container using the following commands from the web

FROM ubuntu:18.04

ENV container docker
ENV PATH /snap/bin:$PATH
ADD snap /usr/local/bin/snap
RUN apt-get update
RUN apt-get install -y snapd squashfuse fuse
RUN systemctl enable snapd
STOPSIGNAL SIGRTMIN+3
CMD [ "/sbin/init" ]

snap script

!/bin/sh -e

while ! kill -0 $(pidof snapd) 2>/dev/null; do
  echo "Waiting for snapd to start."
  sleep 1
done

/usr/bin/snap $@

and building docker

docker build -t snapd .

and run

 docker run --name=snapd -ti -d --tmpfs /run --tmpfs /run/lock --tmpfs /tmp --privileged -v /lib/modules:/lib/modules:ro snapd

up to here everything is fine.

However if try to install microk8s via snap it fails 

snap install microk8s --classic --channel=1.18/stable
2020-04-27T14:22:39Z INFO Waiting for restart...
error: cannot perform the following tasks:
- Run install hook of "microk8s" snap if present (run hook "install": execv failed: Permission denied)

checking snap systemctl status snapd.service gives me 

Apr 27 15:14:32 8985fc7fc5cb snapd[489]: helpers.go:961: cannot retrieve info for snap "microk8s": cannot find installed snap "microk8s" at revision 1341: missing file /sn
ap/microk8s/1341/meta/snap.yaml
Apr 27 15:14:33 8985fc7fc5cb snapd[489]: helpers.go:105: error trying to compare the snap system key: system-key versions not comparable
Apr 27 15:14:33 8985fc7fc5cb snapd[489]: helpers.go:961: cannot retrieve info for snap "microk8s": cannot find installed snap "microk8s" at revision 1341: missing file /sn
ap/microk8s/1341/meta/snap.yaml
Apr 27 15:14:33 8985fc7fc5cb systemd[1]: Started Snappy daemon.
Apr 27 15:15:08 8985fc7fc5cb snapd[489]: handlers.go:495: Reported install problem for "microk8s" as e11fe0c4-8899-11ea-a8e2-fa163ee63de6 OOPSID
CAFEBABE
  • 3,983
  • 1
  • 19
  • 38
  • You might look into [kind](https://kind.sigs.k8s.io/) but in general running something like this inside a container is difficult. (`systemctl` is one of a couple of things that broadly just doesn’t work in Docker and usually you’d want to restructure things to avoid needing an init system and especially systemd.) – David Maze Apr 27 '20 at 15:18
  • @DavidMaze hence the only way is a (k)vm? this is annoying. – CAFEBABE Apr 27 '20 at 15:19
  • @DavidMaze: I also added you recommendation with kind as one of the answers – CAFEBABE Apr 27 '20 at 15:24

2 Answers2

2

I found a (for me) somewhat satisfying answer is to use k3s A German description can be found here

The key is the following docker-compose.yml

version: '3'
services:
  server:
    image: rancher/k3s:v0.8.1
    command: server --disable-agent
    environment:
    - K3S_CLUSTER_SECRET=somethingtotallyrandom
    - K3S_KUBECONFIG_OUTPUT=/output/kubeconfig.yaml
    - K3S_KUBECONFIG_MODE=666
    volumes:
    - k3s-server:/var/lib/rancher/k3s
    # get the kubeconfig file
    - .:/output
    ports:
    - 6443:6443

  node:
    image: rancher/k3s:v0.8.1
    tmpfs:
    - /run
    - /var/run
    privileged: true
    environment:
    - K3S_URL=https://server:6443
    - K3S_CLUSTER_SECRET=somethingtotallyrandom
    ports:
      - 31000-32000:31000-32000

  worker:
    image: rancher/k3s:v0.8.1
    tmpfs:
    - /run
    - /var/run
    privileged: true
    environment:
    - K3S_URL=https://server:6443
    - K3S_CLUSTER_SECRET=somethingtotallyrandom

volumes:
  k3s-server: {}
CAFEBABE
  • 3,983
  • 1
  • 19
  • 38
0

(Just for completeness but I don't accept this answer, if so it my own but suggested by @David Maze) You can use kind Kubernetes in Docker. it was primarily developed for testing docker itself.

CAFEBABE
  • 3,983
  • 1
  • 19
  • 38