Kubernetes - Liveness and Readiness probe implementation

ghz 7months ago ⋅ 166 views

I'm developing a service using Spring and deploying it on OpenShift. Currently I'm using Spring Actuator health endpoint to serve as a liveness and readiness probe for Kubernetes.

However, I will add a call to another service in a Actuator health endpoint, and it looks to me that in that case I need to implement new liveness probe for my service. If I don't do that then a failure in a second service will result with a failure in liveness probe failing and Kubernetes will restart my service without any real need.

Is it OK, for a liveness probe, to implement some simple REST controller which will always return HTTP status 200? If it works, the service can always be considered as alive? Or is there any better way to do it?

Answers

Yes, it's a good idea to distinguish between liveness and readiness probes in your situation. The liveness probe should confirm that your service is running and not in a crashed or deadlocked state, while the readiness probe should check if your service is fully ready to handle requests, including dependencies on other services.

Implementing Liveness and Readiness Probes

Liveness Probe: A liveness probe should be simple and check the health of your service itself, ensuring that it’s alive and running. You can implement a simple REST endpoint that always returns HTTP 200 status if your application is running.

Here’s an example of a simple liveness endpoint:

@RestController
@RequestMapping("/liveness")
public class LivenessController {

    @GetMapping
    public ResponseEntity<String> liveness() {
        return ResponseEntity.ok("OK");
    }
}

In your Kubernetes deployment configuration, you can set this endpoint as the liveness probe:

livenessProbe:
  httpGet:
    path: /liveness
    port: 8080
  initialDelaySeconds: 30
  periodSeconds: 10

Readiness Probe: The readiness probe should check whether your application is ready to serve traffic, which might include checking the health of dependencies such as databases or other services.

Here’s how you might set the readiness probe to use the Spring Actuator health endpoint:

readinessProbe:
  httpGet:
    path: /actuator/health
    port: 8080
  initialDelaySeconds: 30
  periodSeconds: 10

If your Actuator health endpoint includes checks for other services and you don’t want the application to be restarted just because of a dependency failure, make sure that the liveness probe does not rely on those checks.

Separating Health Checks for Probes

You can customize the Actuator health endpoint to separate the checks you want for readiness from those you want for liveness.

  1. Excluding Dependencies from Liveness Checks: You can customize which health indicators are included in the overall health check by configuring the HealthIndicatorRegistry. This way, you can include only lightweight, self-contained checks in your liveness endpoint.

  2. Creating Custom Health Indicators: You can create custom health indicators and register them with Actuator. For example, you might have a DatabaseHealthIndicator for checking the database, which can be included in the readiness probe but excluded from the liveness probe.

Here is an example of customizing health indicators:

@Component
public class MyHealthIndicator implements HealthIndicator {

    @Override
    public Health health() {
        // Custom health check logic
        return Health.up().build();
    }
}

And register it in your health configuration:

@Configuration
public class HealthConfig {

    @Bean
    public HealthIndicator myHealthIndicator() {
        return new MyHealthIndicator();
    }
}

By carefully configuring your liveness and readiness probes, you can ensure that Kubernetes correctly interprets the state of your service and acts appropriately, restarting only when necessary.

#kubernetes #spring