0

I have a kubernetes service that sends messages to a kafka cluster, but sometimes the pods just dont send anything and keep running.

One of my ideas is to monitor the container logs, and when is not sending anything, it alerts me and restart the service. The question is: Is this the best way to do it? And how can i do this? (monitor log and automate restart)

thg1
  • 61
  • 7
  • 2
    You could configure the [liveness probe](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/). For example, you can write a script to check the log and record the healthy. Then you can call this script in your `livenessProbe`. If the `livenessProbe` command returns a non-zero value, the kubelet kills the container and restarts it. – HALF9000 Jul 29 '22 at 02:51
  • 2
    the liveness probe will check the pod health, but I think it is better for you to figure out why have this problem period. restart the pod just hidden the problem. @thg1 – Dolphin Jul 29 '22 at 03:12
  • We can also use the [prometheus monitoring tool](https://devopscube.com/setup-prometheus-monitoring-on-kubernetes/) to monitor the kubernetes cluster . [Refer to this SO](https://stackoverflow.com/questions/53365191/monitor-custom-kubernetes-pod-metrics-using-prometheus?rq=1) – Hemanth Kumar Jul 29 '22 at 09:14

0 Answers0