Automated testing in a build pipeline

A testing mindset with its' scripts would be incomplete if we did not put them to good use.

Ideally, tests should be run every time we intend to share our code with anyone - be it our team or the whole world. It is a tech lead’s wet dream to work in an environment where every dev thoroughly tests their solutions. As this might be impossible, we can have the next best thing.

We can make tests run automatically every time new or modified code is introduced into the codebase.

Let’s assume we do everything in Docker and any new code is built by a server that prevents merging changes until a new image is successfully created.

We’ll use the tests.sh from before and add a test_scenario.sh which checks if all the use cases we need are covered.

# When we run the script:
bash test_scenario.sh

# We get:
Test #1: PASS
Test #2: PASS
Test #3: PASS
Test #4: PASS
Test #5: PASS
Test #6: PASS
Test #7: PASS

Now, we can create a Dockerfile that will use the above script to check if our app is behaving properly.

FROM bash
COPY test*.sh ./

RUN bash test_scenario.sh

The docker image described above will not be built if the tests fail. Like this:

# Try to build the image
docker build .

# We get:

Sending build context to Docker daemon  4.608kB
Step 1/3 : FROM bash
 ---> 78664daf24f4
Step 2/3 : COPY test*.sh ./
 ---> fe75cddc7f0f
Step 3/3 : RUN bash test_scenario.sh
 ---> Running in 010702b02aea
Test #1: PASS
Test #2: PASS
Test #3: PASS
Test #4: PASS
Test #5: PASS
Test #6: PASS
Test #7: FAIL
The command '/bin/sh -c bash test_scenario.sh' returned a non-zero code: 1

With this approach, there will never exist a runnable version of our app that did not pass all the required tests. When we fix the app, the test will work again.

# [...]
Step 3/3 : RUN bash test_scenario.sh
 ---> Running in c2d555481c28
Test #1: PASS
Test #2: PASS
Test #3: PASS
Test #4: PASS
Test #5: PASS
Test #6: PASS
Test #7: PASS
Removing intermediate container c2d555481c28
 ---> d41d9ecb5ed9
Successfully built d41d9ecb5ed9

… and we will have a new image built and ready to use.

Of course, there are different types of tests. Some of them will not be able to run before the application is actually built and deployed to a testing environment. Real-life requires more sophisticated configurations, but the general idea is here.

Both *.sh files and the Dockerfile are available on GitHub Gist .