Kubernetes Health Probes With Spring Boot 101
Kubernetes health probes have just become first class citizens in Spring Boot! Just before the release of Spring Boot 2.3, this article appeared on the Spring blog. These are awesome news because now you can leverage the framework and remove all that custom logic for liveness and readiness checks you have written in the past. In this post, let’s see how you can setup this feature in your application.
Create a demo application
For this tutorial, create a simple spring boot web application that fetches some products. We will create a Product domain entity:
public class Product {
private Long id;
private String description;
private Double price;
public Product(Long id, String description, Double price) {
this.id = id;
this.description = description;
this.price = price;
}
// getters and setters...
}
And a ProductController class:
@RestController
public class ProductController {
private ProductRepository repository;
public ProductController(final ProductRepository repository) {
this.repository = repository;
}
@GetMapping("/product/{id}")
public Product get(@PathVariable final String id) {
return new Product(123L, "Google Pixel 3", 399.99);
}
}
Configure the Kubernetes probes
One very important thing to notice when configuring Kubernetes probes in your Spring Boot application is that they will work out-of-the-box if deployed in a Kubernetes cluster (version 2.3.0+) but when you run from an IDE, they are not auto-configured. You would need to add the following configuration to your application.properties:
management.health.probes.enabled=true
This will register the correct health groups in the Actuator. To verify that it is working as expected, add the config above, run the app from your IDE and query for the /actuator/health endpoint. You should see them like this when the app is running:
{
status: "UP",
groups: [
"liveness",
"readiness",
],
}
Build a container image and Deploy to a Kubernetes cluster
We will use the following Dockerfile to build our image:
FROM openjdk:11.0.2-jre-stretch
COPY target/probesdemo*.jar /opt/app.jar
EXPOSE 8080 5005
ENTRYPOINT exec java -jar /opt/app.jar
We run the build command from the root of the repository where we have our Dockerfile:
$ docker build -t uroshtrifunovic/spring-boot-probes .
For deploying our app in a Minikube cluster, we have the following deployment and service yaml files:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: demo-app
labels:
app: demo-app
spec:
selector:
matchLabels:
app: demo-app
template:
metadata:
labels:
app: demo-app
spec:
containers:
- name: demo-app
image: uroshtrifunovic/spring-boot-probes
imagePullPolicy: Never
ports:
- containerPort: 8080
livenessProbe:
httpGet:
path: /actuator/health/liveness
port: 8080
initialDelaySeconds: 6
periodSeconds: 3
readinessProbe:
httpGet:
path: /actuator/health/readiness
port: 8080
initialDelaySeconds: 5
periodSeconds: 3
resources:
---
apiVersion: v1
kind: Service
metadata:
name: demo-app
labels:
app: demo-app
spec:
type: LoadBalancer
selector:
app: demo-app
ports:
- port: 80
targetPort: 8080
Notice how we configured our liveness and readiness probes in the file to target different endpoints? These endpoints are totally configurable and allow us to put different business logic rules whenever our application is considered unhealthy or unready.
Let’s start our Minikube cluster and create our K8s resources defined in the k8s directory by executing these commands from the project root:
$ minikube start
$ kubectl create -f k8s
If we check our actuator endpoint, we should see that our health groups are registered correctly.
{
status: "UP",
groups: [
"liveness",
"readiness",
],
}
Let’s look at how we can extend this to include a database connection into our readiness healthcheck.
Adding a MySQL database connection
Add the following dependencies in the pom.xml:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<scope>runtime</scope>
</dependency>
Update our Product POJO to an entity, like so:
@Entity
public class Product implements Serializable {
private static final long serialVersionUID = -1679678283409392517L;
@Id
@GeneratedValue(strategy = GenerationType.AUTO)
@Column(name = "id")
private Long id;
@Column(name = "description")
private String description;
@Column(name = "price")
private Double price;
public Product() {
}
public Product(Long id, String description, Double price) {
this.id = id;
this.description = description;
this.price = price;
}
// getters and setters...
Create a Repository class to help us with data access:
@Repository
public interface ProductRepository extends CrudRepository<Product, Long> {
}
And we use it in our ProductController:
@RestController
public class ProductController {
private ProductRepository repository;
public ProductController(final ProductRepository repository) {
this.repository = repository;
}
@GetMapping("/product/{id}")
public Product get(@PathVariable final String id) {
return repository.findById(Long.parseLong(id)).orElseThrow(ProductNotFoundException::new);
}
}
In order to connect to the database, we would need the following configuration in our application.properties:
# Mysql
spring.datasource.driverClassName=com.mysql.jdbc.Driver
# createDatabaseIfNotExist=true is the important part
spring.datasource.url=jdbc:mysql://mysql:3306/product?useSSL=false&autoReconnect=true&useUnicode=true&characterEncoding=UTF-8&createDatabaseIfNotExist=true
spring.datasource.username=root
spring.datasource.password=root
# Hibernate
spring.datasource.platform=mysql
spring.jpa.database-platform=org.hibernate.dialect.MySQL5InnoDBDialect
spring.jpa.hibernate.ddl-auto=update
spring.data.jpa.repositories.enabled=true
spring.datasource.initialization-mode=always
Let’s create K8s resources for our database as well:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: mysql
labels:
app: mysql
spec:
strategy:
type: Recreate
selector:
matchLabels:
app: mysql
template:
metadata:
labels:
app: mysql
spec:
containers:
- name: mysql
image: mysql:5.7.21
ports:
- containerPort: 3306
volumeMounts:
- name: data
mountPath: /var/lib/mysql
env:
- name: MYSQL_ROOT_PASSWORD
value: root
volumes:
- name: data
persistentVolumeClaim:
claimName: mysql
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: mysql
labels:
app: mysql
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
---
apiVersion: v1
kind: Service
metadata:
name: mysql
labels:
app: mysql
spec:
selector:
app: mysql
ports:
- port: 3306
We create our mysql database in the cluster by doing kubectl create -f k8s/mysql and rebuild our application Docker image and redeploy it, so that the changes we just did take effect.
I have inserted some dummy data in the database with this script on startup so we can call our endpoint and verify that the application does indeed connect to the database and obtains the data from there:
insert ignore into product (id, description, price) values (100, 'Google Pixel', 999.99);
insert ignore into product (id, description, price) values (101, 'IPhone', 799.99);
insert ignore into product (id, description, price) values (102, 'Google Nest', 299.99);
At this point, the database will become a dependency to your application. To see more details regarding our health checks, add this config to application.properties:
management.endpoint.health.show-details="ALWAYS"
Rebuild the image and redeploy, then check the /actuator/health endpoint again, you will see that the health of our application is coupled with the health of the database.
{
status: "UP",
components: {
db: {
status: "UP",
details: {
database: "MySQL",
validationQuery: "isValid()",
}
},
livenessState: {
status: "UP"
},
ping: {
status: "UP"
},
readinessState: {
status: "UP"
},
},
groups: [
"liveness",
"readiness",
],
}
At this point, you can move the database dependency to the readiness health group. You can do this by simply adding this configuration to your application.properties:
management.endpoint.health.group.readiness.include=db
If you redeploy this it can be tested by stopping the database pod and see how the application will move into unready state (0/1) but will not crash.
Why should you care about different health endpoints for each probe?
If your application health is coupled to our database and your database is down for whatever reason – that would mean that the health probes were to start failing and restart the application. This is, for sure, undesired behavior as it will trigger all sorts of (false) alarms because there is nothing wrong with the application itself.
On the other hand, if the application cannot use the database, it will get errors when trying to access it and show those errors to the users. That’s why it’s useful to move the database health group under the /actuator/health/readiness endpoint. In case the database is down, only the readinessProbe will fail and simply stop the incoming traffic to the pod. That way, you can fail gracefully and mark the pods as in unready state while you are bringing the database back up. This brings it’s own challenges but depending on your use case, you will want to configure the probes and health groups according to your needs.
Hope you guys liked this tutorial. As always, you can find the code in Github.
Keep on hacking!