How to deploy MinIO object storage in K8s via Helm chart?

Question

As a K8s and MinIO newbie/learner, I would like to enable MinIO Operator in Kubernetes cluster for object storage service via Helm Charts (https://github.com/minio/operator/tree/master/helm, v4.4.1), I have 1 controller/4 nodes. Each node has 4 x 1TB SSDs (each node has /mnt/minio1, /mnt/minio2, /mnt/minio3/, /mnt/mino4). I would like to serve total 16 x 1TB storage space.

Environements:

Ubuntu 20.04.02

Linux am09-17-cyp 5.4.0-65-generic #73-Ubuntu SMP Mon Jan 18 17:25:17 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Deployment CLIs: (*custom values/logs are in the attachement file)

for MinIO Operator, from the controller:

helm install minio-operator ./operator --set accessKey=minioadmin,secretKey=minioadmin123 --set name="minio-operator" --namespace minio-operator --create-namespace --set installCRDs=true -f ./operator_custom_values.yml

for Tenant, from the controller:

helm install minio-tenant-1 ./tenant --set name="minio-tenant-1" --namespace minio-tenant-1 --create-namespace --set installCRDs=true -f ./tenant_custom_values.yml

I have labeled "storage=minio" for 4 nodes
custom values are in the end.

Current questions are:

The custom values files are a little closed to my purpose? (If you have any similar samples to my configuration, it would help me a lot)
How do I config in the custom tenant values to match with the mount points (Currently tenant has /mnt0/data0, /mnt1/data1, .. as below log)?
I have scale up for minio-operator with replicas=4 which didn't work. So changed from Deployment to DaemonSet but the minio-operator pods keep crashing. What would be good practice to balance the load?

kubectl get pods -n minio-operator -o wide

 NAME                              READY   STATUS    RESTARTS   AGE  
 IP              NODE     NOMINATED NODE   READINESS GATES
 console-6c9557b87d-6bbl7          1/1     Running   0          63m  
 x.x.x.x         minio2   <none>           <none>
 minio-operator-5784f97599-96g4q   1/1     Running   0          60m  
 x.x.x.x         minio4   <none>           <none>
 minio-operator-5784f97599-qtd4w   1/1     Running   0          63m  
 x.x.x.x         minio1   <none>           <none>

kubectl scale --replicas=4 deployment minio-operator -n minio-operator

deployment.apps/minio-operator scaled

kubectl get pods -n minio-operator -o wide

 NAME     READY   STATUS    RESTARTS   AGE   IP              NODE     NOMINATED
 NODE   READINESS GATES console-6c9557b87d-6bbl7          1/1    
 Running   0          63m   x.x.x.x         minio2   <none>          
 <none> minio-operator-5784f97599-96g4q   1/1     Running   0         
 60m   x.x.x.x         minio4   <none>           <none>
 minio-operator-5784f97599-fm2gj   1/1     Running   0          4s   
 x.x.x.x         minio1   <none>           <none>
 minio-operator-5784f97599-nftwh   1/1     Running   0          4s   
 x.x.x.x         minio4   <none>           <none>
 minio-operator-5784f97599-qtd4w   1/1     Running   0          63m  
 x.x.x.x         minio1   <none>           <none>

I have also noticed that there is an error in premetheousOperator in the v4.4.1 which I had to comment out to avoid the error:

helm/tenant/templates/tenant.yaml

  # {{- with (dig "readiness" (dict) .) }}
  # readiness:
    # {{ toYaml . | nindent 4 }}
  # {{- end }}
  # {{- with (dig "liveness" (dict) .) }}
  # liveness:
    # {{ toYaml . | nindent 4 }}
  # {{- end }}
  # {{- with (dig "exposeServices" (dict) .) }}
  # exposeServices:
    # {{ toYaml . | nindent 4 }}
  # {{- end }}
  # {{ if dig "serviceAccountName" "" . }}
  # serviceAccountName: {{ dig "serviceAccountName" "" . }}
  # {{ end }}
  # prometheusOperator: {{ dig "prometheusOperator" "false" . }}
  # {{- with (dig "logging" (dict) .) }}
  # logging:
    # {{ toYaml . | nindent 4 }}
  # {{- end }}
  # {{- with (dig "serviceMetadata" (dict) .) }}

The link is the logs and custom values.https://1drv.ms/t/s!ArOQ-I1G8bzlgb8qt8IJA8DlGFijbQ?e=wmiJAd

How to deploy MinIO object storage in K8s via Helm chart?

0 Answers0