how to use nfs storage in crunchydata postgres operator

Question

I am trying out the CrunchyData postgres-operator(Helm) with the NFS Helm chart. And I am unable to create the Cluster with NFS. The following configuration is performed:

Installed NFS helm chart Repo

helm install nfs-abc stable/nfs-server-provisioner

Set the postgres storage values Doc

backrest_storage: 'nfsstorage'
backup_storage: 'nfsstorage'
primary_storage: 'nfsstorage'
replica_storage: 'nfsstorage'

Set the storage configuration Doc

export CCP_SECURITY_CONTEXT='"supplementalGroups": [65534]'
export CCP_STORAGE_PATH=/nfsfileshare
export CCP_NFS_IP=data-nfs-dravoka-nfs-server-provisioner-0.default.svc.cluster.local
export CCP_STORAGE_MODE=ReadWriteMany
export CCP_STORAGE_CAPACITY=400M

Created PGO cluster

pgo create cluster -n pgo dravoka --storage-config='nfsstorage' --pgbackrest-storage-config='nfsstorage' --pvc-size='2Gi'

PVC describe:

kubectl describe -n pgo pvc dravoka
Name:          dravoka
Namespace:     pgo
StorageClass:  standard
Status:        Pending
Volume:
Labels:        pg-cluster=dravoka
               pgremove=true
               vendor=crunchydata
Annotations:   volume.beta.kubernetes.io/storage-provisioner: kubernetes.io/gce-pd
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:
Access Modes:
VolumeMode:    Filesystem
Mounted By:    <none>
Events:
  Type     Reason              Age                    From                         Message
  ----     ------              ----                   ----                         -------
  Warning  ProvisioningFailed  112s (x10 over 7m45s)  persistentvolume-controller  Failed to provision volume with StorageClass "standard": invalid AccessModes [ReadWriteMany]: only AccessModes [ReadWriteOnce ReadOnlyMany] are supported

Pod Describe:

kubectl describe -n pgo pod dravoka-backrest-shared-repo-9fdd77886-j2mjv
Name:           dravoka-backrest-shared-repo-9fdd77886-j2mjv
Namespace:      pgo
Priority:       0
Node:           <none>
Labels:         name=dravoka-backrest-shared-repo
                pg-cluster=dravoka
                pg-pod-anti-affinity=preferred
                pgo-backrest-repo=true
                pod-template-hash=9fdd77886
                service-name=dravoka-backrest-shared-repo
                vendor=crunchydata
Annotations:    <none>
Status:         Pending
IP:
IPs:            <none>
Controlled By:  ReplicaSet/dravoka-backrest-shared-repo-9fdd77886
Containers:
  database:
    Image:      registry.developers.crunchydata.com/crunchydata/pgo-backrest-repo:centos7-4.4.1
    Port:       2022/TCP
    Host Port:  0/TCP
    Requests:
      memory:  48Mi
    Environment:
      PGBACKREST_STANZA:           db
      SSHD_PORT:                   2022
      PGBACKREST_DB_PATH:          /pgdata/dravoka
      PGBACKREST_REPO_PATH:        /backrestrepo/dravoka-backrest-shared-repo
      PGBACKREST_PG1_PORT:         5432
      PGBACKREST_LOG_PATH:         /tmp
      PGBACKREST_PG1_SOCKET_PATH:  /tmp
      PGBACKREST_DB_HOST:          dravoka
    Mounts:
      /backrestrepo from backrestrepo (rw)
      /sshd from sshd (ro)
Conditions:
  Type           Status
  PodScheduled   False
Volumes:
  sshd:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  dravoka-backrest-repo-config
    Optional:    false
  backrestrepo:
    Type:        PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:   dravoka-pgbr-repo
    ReadOnly:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                 node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason            Age                  From               Message
  ----     ------            ----                 ----               -------
  Warning  FailedScheduling  76s (x7 over 9m58s)  default-scheduler  pod has unbound immediate PersistentVolumeClaims (repeated 2 times)

Am I missing something that too is configured or doing something wrong? my purpose is to use NFS as postgres storage. Any help will be appreciated.

Nguyen Lam Phuc · Answer 1 · 2020-09-16T07:47:57.910

0

Failed to provision volume with StorageClass "standard": invalid AccessModes [ReadWriteMany]: only AccessModes [ReadWriteOnce ReadOnlyMany] are supported

So here is the root cause of the problem. You are provisioning the pvc using storage class that does not support the AccessModes that you want i.e. ReadWriteMany

Looking at the doc, seems like you have having this configuration

storage3_name: 'nfsstorage'
storage3_access_mode: 'ReadWriteMany'
storage3_size: '1G'
storage3_type: 'create'
storage3_supplemental_groups: 65534

Where the storage3_access_mode is saying that the access_mode is ReadWriteMany but this is not supported.

Please try to change it to ReadWriteOnce

Anw, Postgres requires block storage to work so even when you get the NFS setup right, your Postgres cluster will not run anyway. More explanation here

edited Sep 16 '20 at 07:47

answered Sep 16 '20 at 04:25

Nguyen Lam Phuc

1,411
1
7
12

PostgreSQL requires a file system, no block storage. NFS will do, but I heard many discouraging voices concerning the reliability of Linux' implementation. – Laurenz Albe Sep 16 '20 at 06:00
Well, at least I cannot get it to work on NFS every single time. So yeah, just some additional information for reference :) – Nguyen Lam Phuc Sep 16 '20 at 06:28
I tried by putting `nfs` as you said, but I am getting an error `"time="2020-09-16T06:40:46Z" level=error msg="Error in pgoconfig: check pgo.yaml: PrimaryStorage setting is invalid: \"nfs\" is not defined" func="internal/apiserver.Initialize()" file="internal/apiserver/root.go:134" version=4.4.1` – Ravindra Gupta Sep 16 '20 at 06:49
Looking at the example here: https://github.com/CrunchyData/postgres-operator/blob/master/conf/postgres-operator/pgo.yaml#L46-L50. I think you should put back `nfsstorage`. And update the `AccessMode` to `ReadWriteOnce`. I will update my answer. – Nguyen Lam Phuc Sep 16 '20 at 07:43

how to use nfs storage in crunchydata postgres operator

1 Answers1