Kubernetes probes with Spring Boot Tutorial

Kubernetes health probes have just become first class citizens in Spring Boot! Just before the release of Spring Boot 2.3, this article appeared on the Spring blog. These are awesome news because now you can leverage the framework and remove all that custom logic for liveness and readiness checks you have written in the past. In this post, let’s see how you can setup this feature in your application.

Create a demo application

For this tutorial, create a simple spring boot web application that fetches some products. We will create a Product domain entity:

public class Product {

    private Long id;
    private String description;
    private Double price;

    public Product(Long id, String description, Double price) {
        this.id = id;
        this.description = description;
        this.price = price;
    }

// getters and setters...
}

And a ProductController class:

@RestController
public class ProductController {

    private ProductRepository repository;

    public ProductController(final ProductRepository repository) {
        this.repository = repository;
    }

    @GetMapping("/product/{id}")
    public Product get(@PathVariable final String id) {
        return new Product(123L, "Google Pixel 3", 399.99);
    }
}

Configure the Kubernetes probes

One very important thing to notice when configuring Kubernetes probes in your Spring Boot application is that they will work out-of-the-box if deployed in a Kubernetes cluster (version 2.3.0+) but when you run from an IDE, they are not auto-configured. You would need to add the following configuration:

management.health.probes.enabled=true

This will register the correct health groups in the Actuator. To verify that it is working as expected, add the config above, run the app from your IDE and query for the /actuator/health endpoint. You should see them like this when the app is running:

{
    status: "UP",
    groups: [
        "liveness",
        "readiness",
    ],
}

Build a container image and
Deploy to a Kubernetes cluster

We will use the following Dockerfile to build our image:

FROM openjdk:11.0.2-jre-stretch

COPY target/probesdemo*.jar /opt/app.jar
EXPOSE 8080 5005

ENTRYPOINT exec java -jar /opt/app.jar

We run the build command from the root of the repository where we have our Dockerfile:

$ docker build -t uroshtrifunovic/spring-boot-probes .

For deploying our app in a Minikube cluster, we have the following deployment and service yaml files:

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: demo-app
  labels:
    app: demo-app
spec:
  selector:
    matchLabels:
      app: demo-app
  template:
    metadata:
      labels:
        app: demo-app
    spec:
      containers:
        - name: demo-app
          image: uroshtrifunovic/spring-boot-probes
          imagePullPolicy: Never
          ports:
            - containerPort: 8080
          livenessProbe:
            httpGet:
              path: /actuator/health/liveness
              port: 8080
            initialDelaySeconds: 6
            periodSeconds: 3
          readinessProbe:
            httpGet:
              path: /actuator/health/readiness
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 3
          resources:
---
apiVersion: v1
kind: Service
metadata:
  name: demo-app
  labels:
    app: demo-app
spec:
  type: LoadBalancer
  selector:
    app: demo-app
  ports:
    - port: 80
      targetPort: 8080

Notice how we configured our liveness and readiness probes in the file to target different endpoints? These endpoints are totally configurable and allow us to put different business logic rules whenever our application is considered unhealthy or unready.

Let’s start our Minikube cluster and create our K8s resources defined in the k8s directory by executing these commands from the project root:

$ minikube start
$ kubectl create -f k8s

If we check our actuator endpoint, we should see that our health groups are registered correctly.

{
  status: "UP",
  groups: [
    "liveness",
    "readiness",
  ],
}

Let’s look at how we can extend this to include a database connection into our readiness healthcheck.

Adding a MySQL database connection

Add the following dependencies in the pom.xml:

		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-data-jpa</artifactId>
		</dependency>
		<dependency>
			<groupId>mysql</groupId>
			<artifactId>mysql-connector-java</artifactId>
			<scope>runtime</scope>
		</dependency>

Update our Product POJO to an entity, like so:

@Entity
public class Product implements Serializable {

    private static final long serialVersionUID = -1679678283409392517L;
    @Id
    @GeneratedValue(strategy = GenerationType.AUTO)
    @Column(name = "id")
    private Long id;

    @Column(name = "description")
    private String description;

    @Column(name = "price")
    private Double price;

    public Product() {
    }

    public Product(Long id, String description, Double price) {
        this.id = id;
        this.description = description;
        this.price = price;
    }

// getters and setters...

Create a Repository class to help us with data access:

@Repository
public interface ProductRepository extends CrudRepository<Product, Long> {
}

And we use it in our ProductController:

@RestController
public class ProductController {

    private ProductRepository repository;

    public ProductController(final ProductRepository repository) {
        this.repository = repository;
    }

    @GetMapping("/product/{id}")
    public Product get(@PathVariable final String id) {
        return repository.findById(Long.parseLong(id)).orElseThrow(ProductNotFoundException::new);
    }
}

In order to connect to the database, we would need the following configuration in our application.properties:

# Mysql
spring.datasource.driverClassName=com.mysql.jdbc.Driver
# createDatabaseIfNotExist=true is the important part
spring.datasource.url=jdbc:mysql://mysql:3306/product?useSSL=false&autoReconnect=true&useUnicode=true&characterEncoding=UTF-8&createDatabaseIfNotExist=true
spring.datasource.username=root
spring.datasource.password=root
# Hibernate
spring.datasource.platform=mysql
spring.jpa.database-platform=org.hibernate.dialect.MySQL5InnoDBDialect
spring.jpa.hibernate.ddl-auto=update
spring.data.jpa.repositories.enabled=true
spring.datasource.initialization-mode=always

Let’s create K8s resources for our database as well:

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mysql
  labels:
    app: mysql
spec:
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: mysql
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
        - name: mysql
          image: mysql:5.7.21
          ports:
            - containerPort: 3306
          volumeMounts:
            - name: data
              mountPath: /var/lib/mysql
          env:
            - name: MYSQL_ROOT_PASSWORD
              value: root
      volumes:
        - name: data
          persistentVolumeClaim:
            claimName: mysql
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mysql
  labels:
    app: mysql
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
---
apiVersion: v1
kind: Service
metadata:
  name: mysql
  labels:
    app: mysql
spec:
  selector:
    app: mysql
  ports:
    - port: 3306

We create our mysql database in the cluster by doing kubectl create -f k8s/mysql and rebuild our application Docker image and redeploy it, so that the changes we just did take effect.

I have inserted some dummy data in the database with this script on startup so we can call our endpoint and verify that the application does indeed connect to the database and obtains the data from there:

insert ignore into product (id, description, price) values (100, 'Google Pixel', 999.99);
insert ignore into product (id, description, price) values (101, 'IPhone', 799.99);
insert ignore into product (id, description, price) values (102, 'Google Nest', 299.99);

At this point, the database will become a dependency to your application. To see more details regarding our health checks, add this config to application.properties:

management.endpoint.health.show-details="ALWAYS"

Rebuild the image and redeploy, then check the /actuator/health endpoint again, you will see that the health of our application is coupled with the health of the database.

{
  status: "UP",
  components: {
    db: {
      status: "UP",
      details: {
        database: "MySQL",
        validationQuery: "isValid()",
      }
    },
    livenessState: {
      status: "UP"
    },
    ping: {
      status: "UP"
    },
    readinessState: {
      status: "UP"
    },
  },
  groups: [
    "liveness",
    "readiness",
  ],
}

At this point, you can move the database dependency to the readiness health group. You can do this by simply adding this configuration to your application.properties:

management.endpoint.health.group.readiness.include=db

If you redeploy this it can be tested by stopping the database pod and see how the application will move into unready state (0/1) but will not crash.

Why should you care about different health endpoints for each probe?

If your application health is coupled to our database and your database is down for whatever reason – that would mean that the health probes were to start failing and restart the application. This is, for sure, undesired behavior as it will trigger all sorts of (false) alarms because there is nothing wrong with the application itself.

On the other hand, if the application cannot use the database, it will get errors when trying to access it and show those errors to the users. That’s why it’s useful to move the database health group under the /actuator/health/readiness endpoint. In case the database is down, only the readinessProbe will fail and simply stop the incoming traffic to the pod. That way, you can fail gracefully and mark the pods as in unready state while you are bringing the database back up. This brings it’s own challenges but depending on your use case, you will want to configure the probes and health groups according to your needs.

Hope you guys liked this tutorial. As always, you can find the code in Github.
Also make sure to check out other tutorials.

Keep on hacking!

About Author


Urosh Trifunovic

Leave a Reply

Your email address will not be published. Required fields are marked *