Mount RDB volumes in kubernetes + coreos

Question

I'm trying to mount a ceph RDB volume in a pod create by kubernetes under CoreOS.

CoreOS version is beta (1153.4.0)
Hyperkube version is v1.3.7+coreos.0
ceph version is jewel

Is use this POD:

{
  "apiVersion": "v1",
  "kind": "Pod",
  "metadata": {
    "name": "rbd2"
  },
  "spec": {
    "containers": [
      {
        "name": "rbd-rw",
        "image": "nginx",
        "volumeMounts": [
          {
            "mountPath": "/mnt/rbd",
            "name": "rbdpd"
          }
        ]
      }
    ],
    "volumes": [
      {
        "name": "rbdpd",
        "rbd": {
          "monitors": [
            "10.203.69.254"
          ],
          "pool": "rbd",
          "image": "foo",
          "user": "admin",
          "secretRef": {
            "name": "ceph-secret"
          },
          "fsType": "ext4",
          "readOnly": true
        }
      }
    ]
  }
}

If i start this pod it is created but stay in status: ContainerCreating

The problem is that the pod seems not to be able to create the rbd device.

If i connect to the node where the pod has been deployed and create the rbd device (on the host) with the rbdmap command, the device is create and the pod can mount the volume. Then the pod's status switch to Running

If i delete the pod, the rbd device is removed automatically.

To resume, what can be the issue that cause kubernetes to not be able to create the rbd device.

score 1 · Answer 1 · answered Sep 22 '16 at 17:14

If you are using the CoreOS kubelet-wrapper script to launch the kubelet, you should be able to use rbd devices by adding a few extra mount points.

An example override of the RKT_OPTS for the kubelet-wrapper:

[Service]
Environment="RKT_OPTS=--volume modprobe,kind=host,source=/usr/sbin/modprobe \
--mount volume=modprobe,target=/usr/sbin/modprobe \
--volume lib-modules,kind=host,source=/lib/modules \
--mount volume=lib-modules,target=/lib/modules \
Environment=KUBELET_VERSION=v1.3.7_coreos.0
...

Source: kubelet-wrapper rbd docs.

Mount RDB volumes in kubernetes + coreos

1 Answers1