0

How could I debug a pod stuck in pending state? I am using k8ssandra https://k8ssandra.io/docs/ to create a Cassandra cluster. It uses helm files. I created a 3 nodes cluster and changed size value to 3 in local values.yaml file to create a 3 node cluster - https://github.com/k8ssandra/k8ssandra/blob/main/charts/k8ssandra-cluster/values.yaml

no_reply@cloudshell:~ (k8ssandra-299315)$ kubectl get pods
NAME                                                              READY   STATUS      RESTARTS   AGE
cass-operator-86d4dc45cd-588c8                                    1/1     Running     0          29h
grafana-deployment-66557855cc-j7476                               1/1     Running     0          29h
k8ssandra-cluster-a-grafana-operator-k8ssandra-5b89b64f4f-8pbxk   1/1     Running     0          29h
k8ssandra-cluster-a-reaper-k8ssandra-847c99ccd8-dsnj4             1/1     Running     0          28h
k8ssandra-cluster-a-reaper-k8ssandra-schema-5fzpn                 0/1     Completed   0          28h
k8ssandra-cluster-a-reaper-operator-k8ssandra-87d56d56f-wn8hw     1/1     Running     0          29h
k8ssandra-dc1-default-sts-0                                       2/2     Running     0          29h
**k8ssandra-dc1-default-sts-1                                       0/2     Pending     0          14m**
k8ssandra-dc1-default-sts-2                                       2/2     Running     0          14m
k8ssandra-tools-kube-prome-operator-6bcdf668d4-ndhw9              1/1     Running     0          29h
prometheus-k8ssandra-cluster-a-prometheus-k8ssandra-0             2/2     Running     1          29h
Aaron
  • 55,518
  • 11
  • 116
  • 132
Manu Chadha
  • 15,555
  • 19
  • 91
  • 184
  • 1
    Describe the pod and check events – Arghya Sadhu Dec 23 '20 at 14:26
  • Thanks. Can see `Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning FailedScheduling 33m (x3 over 33m) default-scheduler pod has unbound immediate PersistentVolumeClaims Warning FailedScheduling 49s (x24 over 33m) default-scheduler 0/3 nodes are available: 3 Insufficient cpu.`. Now need to figure out what went wrong :) ` – Manu Chadha Dec 23 '20 at 14:40
  • the storage class is already marked as default. The duplicate question doesn't solve my problem (though the original question about debugging has been answered) – Manu Chadha Dec 23 '20 at 15:40

1 Answers1

1

The best way as described by Arghya is checking the events of the pod.

kubectl describe pod k8ssandra-dc1-default-sts-1

You could also check for the logs of the pod:

kubectl logs k8ssandra-dc1-default-sts-1
CaioT
  • 1,973
  • 1
  • 11
  • 20
  • Thanks. Tried logs but got message `error: a container name must be specified for pod k8ssandra-dc1-default-sts-1, choose one of: [cassandra server-system-logger] or one of the init containers: [server-config-init jmx-credentials]`. Then checked for each container but don't see any logs. – Manu Chadha Dec 23 '20 at 14:42
  • the log commands are: kubectl logs -n cass-operator cluster1-dc1-default-sts-0 -c server-config-init kubectl logs -n cass-operator cluster1-dc1-default-sts-0 -c server-system-logger kubectl logs -n cass-operator cluster1-dc1-default-sts-0 -c cassandra – DS Steven Matison Jan 04 '21 at 12:48