0

I'm testing with a small k8s cluster and ceph cluster to create and assign persistent volumes. Ceph cluster version is Nautilus and the machines in k8s cluster are ubuntu 20.04. As far as I understand, I can't install ceph nautilus common packages on Ubuntu 20.04. Is it possible to install Ceph Octopus common packages on the machines and connect them to a Nautilus cluster?

Nyquillus
  • 179
  • 1
  • 5
  • 23
  • Can't you use Rook ( https://rook.io/ ) to install Ceph inside the Kubernetes cluster as Pods ? It can install Ceph whole cluster-infrastructure as an in-containers-thing.. as long as you assign it empty disks from the hosts. Or you are trying to connect the kubernetes cluster with an external ceph cluster, located on other machines? – AndD Jun 30 '21 at 08:28
  • The ceph cluster is seperate from the kubernetes cluster yes. What kind of an approach would you suggest for that? – Nyquillus Jun 30 '21 at 09:33
  • 1
    I NEVER tried this solution, so I am not sure this is what I would go for.. but in theory you could use Rook to install inside the cluster only the drivers to access the external ceph cluster as dynamic storage on your Kubernetes cluster. This is explained here: https://github.com/rook/rook/blob/master/design/ceph/ceph-external-cluster.md and have examples on github: https://github.com/rook/rook/tree/master/cluster/examples/kubernetes/ceph – AndD Jun 30 '21 at 09:41
  • So if i go with this method i won't need to install ceph-common packages to the k8s cluster right? – Nyquillus Jun 30 '21 at 10:48
  • 1
    Yes, from my understanding there's no need for that. The Rook operator will instead install the csi-driver and the other things necessary to connect to the external cluster as Pods inside the Kuberneets cluster itself. Once this is done, rbd can be provisioned on Kubernetes dynamically, plus there should be support for the ceph filesystem and s3. I've found another example here: https://rook.io/docs/rook/v1.1/ceph-cluster-crd.html#external-cluster – AndD Jun 30 '21 at 11:11

2 Answers2

2

Yes, you can connect to a nautilus cluster with a client in versions of both octopus and pacific.

Seena Fallah
  • 560
  • 4
  • 12
1

Edit: nevermind, I was wrong.

It depends of what you intend to do.

Obviously yes, an octopus client can connect to a nautilus cluster.

Though if you're looking to setup dynamic volume provisioning, nowadays you would need CSI (ceph-csi on github). And then, I think you need an octopus cluster at least.

SYN
  • 4,476
  • 1
  • 20
  • 22
  • The support chart on the github page says csi supports nautilus versions and above is it false information? – Nyquillus Jul 05 '21 at 06:42
  • Indeed, that's weird. Though I'm pretty sure I ended up upgrading to Octopus because of ceph-csi. Using RBD, on Kubernetes, which is my only use for Ceph ATM. I can't remember exactly, it was a while ago: something about setting a feature on the pool hosting my rbd volumes. Sadly, I don't have another ceph cluster to test that with. – SYN Jul 05 '21 at 07:04
  • I'll comment here again after my testing is done with nautilus. Hope I won't need to upgrade my cluster for now. – Nyquillus Jul 05 '21 at 08:22
  • I connected to the cluster with ceph csi, no problems so far. However at first i created a client specifically for kubernetes on ceph but i got permission denied errors when deploying pods with that user. So after some digging i found out it was about the latest version on ceph csi and they recommend to use an older version or client.admin credentials. – Nyquillus Jul 05 '21 at 12:18
  • 1
    Nevermind then, thanks for checking it out! – SYN Jul 06 '21 at 16:21