Java Integration Test with Dockerized DB in Jenkins

• Download source - 69.1 KB

Introduction

This article explains how to setup integration test in jenkins using dockerized database. It is important that we should execute the integration test in CI pipeline so that we get hold of any error in the CI itself. Since integration test depends on database and other services, we need to have those installed in CI. If your application has lot of dependencies, installing those particular versions is going to be tedious. But going with docker, it simplifies those dependent things. This article explains how to do it with dockerized database.

Disclaimer

This uses a plain script based way to run the integration test in Jenkins. This uses the Spring project and junit 5 testcase for demonstration purposes. Since it mostly deals with testcase and coverage, there is no controller or API as such.

Also, this article mostly concentrates on setting up integration test and dockerized database setup along with coverage feedback. I am planning to add an article explaining how to write integration testcases.

Pre-setup

Now, we will jump to the pre-setup part. I have taken a spring boot based project. It is a simple employee CRUD application. It has data access layer and service layer. It contains certain validation such as the following:

• Employee email should be in the correct format.
• While updating employee, it should get the existing employee id.
• While updating, it cannot use an existing email of another employee.
• Non-existent employee id can't be deleted.

All those test cases are written using new JUnit 5. I have already explained about certain important features of jUnit 5 here. I have written the integration test by covering the above cases. Here is the link to the source code.

Setup

Docker can be installed using docker image or standalone installer. Here, we are picking standalone installer, as we are planning to run the dependent services on docker. As it might be problematic to setup docker inside docker, I have not taken jenkins as docker image.

Jenkins can be downloaded from here - https://jenkins.io/download/.

After installing jenkins, create a new job with name "java-int-test" and choose freestyle project. Here is how it looks like:

Jenkins job has different sections such as SCM, Build frequency, Build Commands, Post-Build setup. I will explain those in details.

SCM: It is expected to give path to the source code. Since we are going to build and run testcase, it is expected to have some source code to start with. As we are using github, I have chosed git. If it is private repo, you have to give credential, so that it can fetch the source code. Here is how the setting looks like:

Build Frequency: Here, we can specify when we should build the project. I have chosen to build it hourly. We can also set it up to run on every commit to the repo. Here is my setting:

Build Commands: This is the crucial step, where we specify what are the commands to execute against the source code we have taken. Here is how my setting looks like:

Since we are running dockerized db, we need to install docker in the system where jenkins is running. Also, we are using mariadb (mysql variant) as db for integration test, so we need to have mysql client installed in the same machine as well, because we need to apply db scripts before the test runs.

Here are those lines:

docker stop testdb || true
docker rm testdb || true
docker run --name testdb -p 4306:3306 -e MYSQL_ROOT_PASSWORD=root -d mariadb/server:10.3
/bin/sleep 30s
mysql --host=127.0.0.1 --port=4306 --user="root" -proot 
      --execute="CREATE SCHEMA IF NOT EXISTS demo_test;"
mysql --host=127.0.0.1 --port=4306 --user="root" 
      -proot demo_test < src/main/resources/migration/V1.1__create_employee_table.sql
mysql --host=127.0.0.1 --port=4306 --user="root" 
      -proot demo_test < src/main/resources/migration/V1.2__seed_employees.sql
mysql --host=127.0.0.1 --port=4306 --user="root" -proot -e "select * from demo_test.Employee"
./gradlew clean build test

I will explain the above commands below:

• For now, ignore the first 2 lines, will explain them later.
• 3^rd line, here we are pulling mariadb server with version as 10.3 with port mapping of 4306 of host and mysql root password being set to root. This creates a container with name as testdb.
• Since the container initialize will take some time, we have to wait till it comes up, so I put a sleep of 30s.
• Now our db is up and running, in 4^th-7^th line, we are connecting to DB for various activities, i.e., create blank schema, apply db scripts and verify if everything is up and running. If you notice, to connect to docker container DB, we are using mysql --host=127.0.0.1 --port=4306 --user="root" -proot, here we are utilizing the port and credential as root / root, which we setup in the 3^rd line.
• In the last line, we are building our project and running the tests.
• The first 2 lines, actually do the clean up of container. So that our test is repeatable, for every run, we will get a clean container with clean DB.

In test application properties, it points to the test database. Here is the setting file:

# ===============================
# = DATA SOURCE
# ===============================
# Set here configurations for the database connection
spring.datasource.url=jdbc:mariadb://localhost:4306/demo_test
spring.datasource.username=root
spring.datasource.password=root
spring.datasource.driver-class-name=org.mariadb.jdbc.Driver
# Keep the connection alive if idle for a long time (needed in production)
spring.datasource.testWhileIdle=true
spring.datasource.validationQuery=SELECT 1

Coverage Setup

I have used jacoco gradle plugin to get the coverage report.

id 'jacoco'

test {
    useJUnitPlatform()
    finalizedBy jacocoTestReport
}

To publish coverage report in jenkins, I have used plugins - "JUnit Realtime Test Reporter Plugin" , "JaCoCo Plugin". To install it, go to "Manage Jenkins" --> "Manage Plugins" --> click on "Available" tab --> search for above 2. Then select those and install. This requires jenkins restart, please proceed with that.

The next step is to configure those plugins to read our test report and nicely publish them to jenkins.

Here is the setting for Junit test XML. Since our test XMLs are created under `build/test-results/test/', I have given that path there.

Here is the setting for Jacoco. I have not made any change there, it is the default one.

After setting all these up, navigate to the job we created and click on "build now" on the left side. After that, you see a build number like the following:

Click on the recent build number, then click on "Console output". There, you can see all those commands running. If everything is successful, then it should show in blue like the above screenshot.

After successful run, here is how the coverage report looks like. It will give you all types of coverages like class, line, method, etc.

And here is the unit test report.

You can browse through all the testcase execution from here. And you can see what all remaining lines are there to cover, etc.

Points of Interest

There are a lot of things that can be done for coverage and integration test. I am listing those. You can add a lot more dependency to the jenkins build process if your application depends on those. E.g., database, cache, solr, queueing service, etc. In terms of coverage, gradle provides lot of options like mark the build as failure if certain % of coverage is not met and there can be a lot more to it.

History

This is my initial published content. I will keep the list of revisions here.

• Revision 1: Initial publication
• Revision 2: Added jenkins and dockerized db setup