Pod Health Probing

In order to verify if a container in a pod is healthy and ready to serve traffic, Kubernetes provides for a range of health checking mechanisms. Health checks, or probes as they are called in Kubernetes, are carried out by the kubelet to determine when to restart a container (for livenessProbe) and by services to determine if a pod should receive traffic or not (for readinessProbe).

We will focus on HTTP health checks in the following. Note that it is the responsibility of the application developer to expose a URL that the kubelet can use to determine if the container is healthy (and potentially ready).

Let’s create a pod that exposes an endpoint /health, responding with a HTTP 200 status code:

kubectl create -f https://raw.githubusercontent.com/mhausenblas/kbe/master/specs/healthz/pod.yaml

In the pod specification we’ve defined the following:

livenessProbe:

  initialDelaySeconds: 2

  periodSeconds: 5

  httpGet:

    path: /health

    port: 9876

Above means that Kubernetes will start checking /health endpoint in every 5 seconds after waiting 2 seconds for the first check.

If we now look at the pod we can see that it is considered healthy:

kubectl describe pod hc

Now we launch a bad pod, that is, a pod that has a container that randomly (in the time range 1 to 4 sec) does not return a 200 code:

kubectl create -f https://raw.githubusercontent.com/mhausenblas/kbe/master/specs/healthz/badpod.yaml

Looking at the events of the bad pod, we can see that the health check failed:

kubectl describe pod badpod

This can also be verified as follows:

kubectl get pods

From above you can see that the badpod had already been re-launched 4 times, since the health check failed.

In addition to a livenessProbe you can also specify a readinessProbe, which can be configured in the same way but has a different use case and semantics: it’s used to check the start-up phase of a container in the pod. Imagine a container that loads some data from external storage such as S3 or a database that needs to initialize some tables. In this case you want to signal when the container is ready to serve traffic.

Let’s create a pod with a readinessProbe that kicks in after 10 seconds:

kubectl create -f https://raw.githubusercontent.com/mhausenblas/kbe/master/specs/healthz/ready.yaml

Looking at the events of the pod, we can see that, eventually, the pod is ready to serve traffic:

kubectl describe pod ready

You can remove all the created pods with:

kubectl delete pod/hc pod/ready pod/badpod

<< Previous                                                                                                                          Next  >>