I'm facing an issue with redis enterprise, anyone can help me? I followed the docs but I'm stuck at this point: after the apply command I get only "Back-off restarting failed container" error. Logs from the pod return output only for the admission container, the only running. No logs return from redis-enterprise-operator container, the one in the CrashLoopBackOff state.
What am I missing?
I tried the command:
kubectl apply -f https://raw.githubusercontent.com/RedisLabs/redis-enterprise-k8s-docs/v6.4.2-4/bundle.yaml -n redis
both in Minikube (with k8s 1.25 and 1.24) and in EKS (with k8s 1.25), and this is the outuput:
role.rbac.authorization.k8s.io/redis-enterprise-operator created
rolebinding.rbac.authorization.k8s.io/redis-enterprise-operator created
serviceaccount/redis-enterprise-operator created
service/admission created
customresourcedefinition.apiextensions.k8s.io/redisenterpriseclusters.app.redislabs.com configured
customresourcedefinition.apiextensions.k8s.io/redisenterprisedatabases.app.redislabs.com configured
deployment.apps/redis-enterprise-operator created
Checking resources, this is the state:
kubectl get all -n redis
NAME READY STATUS RESTARTS AGE
pod/redis-enterprise-operator-59d9688459-mcqdt 1/2 CrashLoopBackOff 5 (113s ago) 8m17s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/admission ClusterIP 10.105.61.55 <none> 443/TCP 8m59s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/redis-enterprise-operator 0/1 1 0 8m18s
NAME DESIRED CURRENT READY AGE
replicaset.apps/redis-enterprise-operator-59d9688459 1 1 0 8m18s
I read here the supported k8s versions and it seems Minikube can support the image I've chosen. On the other hand EKS is not yet supported with k8s 1.25 so a kind of error was expected. The problem is that it occurs even on Minikube. So, is this the actual issue? Is there anything else wrong?
EDIT: I paste here also the output of this command:
kubectl get pod/redis-enterprise-operator-59d9688459-mcqdt -o yaml -n redis
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: "2023-03-30T07:30:56Z"
generateName: redis-enterprise-operator-59d9688459-
labels:
app: redis-enterprise
name: redis-enterprise-operator
pod-template-hash: 59d9688459
name: redis-enterprise-operator-59d9688459-mcqdt
namespace: redis
ownerReferences:
- apiVersion: apps/v1
blockOwnerDeletion: true
controller: true
kind: ReplicaSet
name: redis-enterprise-operator-59d9688459
uid: bf58858c-8e0b-4adc-b7ac-bb0b9f0a9b90
resourceVersion: "1879"
uid: cb1068a2-3e2d-4625-a175-faeda18f45bd
spec:
containers:
- command:
- redis-enterprise-operator
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
- name: WATCH_NAMESPACE
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
- name: POD_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.name
- name: OPERATOR_NAME
value: redis-enterprise-operator
envFrom:
- configMapRef:
name: operator-environment-config
optional: true
image: redislabs/operator:6.4.2-4
imagePullPolicy: Always
livenessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 8080
scheme: HTTP
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
name: redis-enterprise-operator
ports:
- containerPort: 8080
protocol: TCP
resources:
limits:
cpu: "4"
memory: 512Mi
requests:
cpu: 500m
memory: 256Mi
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: kube-api-access-46d6v
readOnly: true
- command:
- /usr/local/bin/admission
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
- name: WATCH_NAMESPACE
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
envFrom:
- configMapRef:
name: operator-environment-config
optional: true
image: redislabs/operator:6.4.2-4
imagePullPolicy: Always
livenessProbe:
failureThreshold: 3
httpGet:
path: /liveness
port: 8443
scheme: HTTPS
initialDelaySeconds: 15
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
name: admission
ports:
- containerPort: 8443
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 8443
scheme: HTTPS
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 5
resources:
limits:
cpu: "1"
memory: 512Mi
requests:
cpu: 250m
memory: 256Mi
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: kube-api-access-46d6v
readOnly: true
dnsPolicy: ClusterFirst
enableServiceLinks: true
nodeName: minikube
preemptionPolicy: PreemptLowerPriority
priority: 0
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
serviceAccount: redis-enterprise-operator
serviceAccountName: redis-enterprise-operator
terminationGracePeriodSeconds: 30
tolerations:
- effect: NoExecute
key: node.kubernetes.io/not-ready
operator: Exists
tolerationSeconds: 300
- effect: NoExecute
key: node.kubernetes.io/unreachable
operator: Exists
tolerationSeconds: 300
volumes:
- name: kube-api-access-46d6v
projected:
defaultMode: 420
sources:
- serviceAccountToken:
expirationSeconds: 3607
path: token
- configMap:
items:
- key: ca.crt
path: ca.crt
name: kube-root-ca.crt
- downwardAPI:
items:
- fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
path: namespace
status:
conditions:
- lastProbeTime: null
lastTransitionTime: "2023-03-30T07:30:56Z"
status: "True"
type: Initialized
- lastProbeTime: null
lastTransitionTime: "2023-03-30T07:30:56Z"
message: 'containers with unready status: [redis-enterprise-operator]'
reason: ContainersNotReady
status: "False"
type: Ready
- lastProbeTime: null
lastTransitionTime: "2023-03-30T07:30:56Z"
message: 'containers with unready status: [redis-enterprise-operator]'
reason: ContainersNotReady
status: "False"
type: ContainersReady
- lastProbeTime: null
lastTransitionTime: "2023-03-30T07:30:56Z"
status: "True"
type: PodScheduled
containerStatuses:
- containerID: docker://5d6ab0bd54ed526329df3e6790059b48ef85014d2f9aab7d135257815d110806
image: redislabs/operator:6.4.2-4
imageID: docker-pullable://redislabs/operator@sha256:6f9c23fb667cd9c556a04d33de15aa4903028891db10e350dc8d787546c0ea37
lastState:
terminated:
containerID: docker://594bf56135b0a7392cf59be67e14b94bc5a8a631d0b58b172b92d57740684896
exitCode: 137
finishedAt: "2023-03-30T08:30:30Z"
reason: Error
startedAt: "2023-03-30T08:25:08Z"
name: admission
ready: true
restartCount: 8
started: true
state:
running:
startedAt: "2023-03-30T08:30:39Z"
- containerID: docker://ddd30597ed84a3509f684401f71db3397c9491b77e6695199de18ad1ccf102d5
image: redislabs/operator:6.4.2-4
imageID: docker-pullable://redislabs/operator@sha256:6f9c23fb667cd9c556a04d33de15aa4903028891db10e350dc8d787546c0ea37
lastState:
terminated:
containerID: docker://ddd30597ed84a3509f684401f71db3397c9491b77e6695199de18ad1ccf102d5
exitCode: 128
finishedAt: "2023-03-30T08:28:55Z"
message: 'failed to create shim task: OCI runtime create failed: runc create
failed: unable to start container process: error during container init:
error setting cgroup config for procHooks process: failed to write "400000":
write /sys/fs/cgroup/cpu/kubepods/burstable/podcb1068a2-3e2d-4625-a175-faeda18f45bd/ddd30597ed84a3509f684401f71db3397c9491b77e6695199de18ad1ccf102d5/cpu.cfs_quota_us:
invalid argument: unknown'
reason: ContainerCannotRun
startedAt: "2023-03-30T08:28:55Z"
name: redis-enterprise-operator
ready: false
restartCount: 15
started: false
state:
waiting:
message: back-off 5m0s restarting failed container=redis-enterprise-operator
pod=redis-enterprise-operator-59d9688459-mcqdt_redis(cb1068a2-3e2d-4625-a175-faeda18f45bd)
reason: CrashLoopBackOff
hostIP: 192.168.58.2
phase: Running
podIP: 10.244.0.3
podIPs:
- ip: 10.244.0.3
qosClass: Burstable
startTime: "2023-03-30T07:30:56Z"
