How to use TestBox results within GitLab CI pipeline

In this post, we are going to see what TestBox is, why you should work with it, and how you can integrate it with Gitlab. We all know that Coldfusion is a small community but still there is enough information out there to make things great!.

Development Coldfusion

27 oktober 2020

In this post, we are going to see what TestBox is, why you should work with it, and how you can integrate it with Gitlab. We all know that Coldfusion is a small community but still there is enough information out there to make things great!.

What is TestBox

Testbox is a testing framework for ColdFusion (CFML) that is based on BDD unit testing. If you are not familiar with this concept, there are two kinds of unit testing:

• TDD (Test Driven Development) the test is written to check the implementation of functionality, but as the code evolves, tests can give false results.

• BDD (Behavior Driven Development) is also a test-first approach but differs by testing the actual behavior of the system from the end-users perspective.

You can say that TDD gives you the when and BDD gives you the how, they can be used individually or you can combine them for best results as they complement each other very nicely.

 

TestBox contains not only a testing framework, runner, assertions, mocking, and expectations library but it also supports xUnit style of testing.

Why should I use it?

These four points are the main reasons for using TestBox:

1. Refactoring is safer. Refactoring mandates that each change is tested immediately to verify the accomplishment of the desired goal.

2. Improve the quality and reliability of our application.

3. Faster finding bugs when you make changes to your code.

4. Better deployment during your build process.

How to install and use TestBox on his own? 

First of all, we will need to install TestBox on our application. You can easily follow the steps here. But if you don’t want to read more documentation than necessary, just launch this script using your CommandBox on your application directory:

 

box install testbox --saveDev

 

Once installed, you can find a test folder in the your webroot directory we can navigate to the URL: localhost/test/runner.cfm or just launch the testbox run command from our Commandbox to access the testBox console.

 

Here you have plenty of information on how to work with TestBox but I'm going to provide a tip that I believe is going to help:

TestBox has the functionality of using Code-Coverage that provides additional metrics gathered by the testing framework while your tests are running that actually tracks what lines of code were executed and what lines of code didn't get run. Now you can finally see how much code in your app is "covered" by your tests and what code is currently being untested which makes this functionality pretty handy.

 

Now make sure you have enabled this functionality on your runner.cfm file like this:

<cfparam name="url.coverageEnabled"         default="true">

And you will be able to see the functionality on the TestBox panel:

Next step is prepare your spect test suits and to have a workable TDD setup where all the user cases should match with one test suit.

• As user I should be able to login in the portal

• As user I should be able to get the endpoint /entity/blog

• As user I should be able to update the entity blog via api endpoint PUT entity/blog

• …...

What are the advantages of using TestBox with Gitlab CI Pipeline?

Here you find a list of the advantages of using Textbox with Gitlab:

 

• Repeatable.

• Can be run at the push of a button. 

  • OneTest

  • A suite of tests

  • All unit tests

• Works with any CI, not just Gitlab but also Jenkins or Ant.

• Consistent test results independent of the environment (eg database).

• Full coverage of all code in the unit.

• Independent of other tests – can be run in random order – no dependencies between tests.

• Easy to detect if an error occurs and report on it so you can.

• Test setup and tear down of dependencies.

How can I use TestBox within Gitlab?

CI/CD is a good place to run the test suits. As other stages we can create a pipeline stage which will run the test when we merge to any specific branch or create a tag.

 

Gitlab also provides a test integration where we can send the test result in a specific format and Gitlab can show the test result in a beautiful and fancy design which provides the user a nice overview about the test result.

 

The way to implement it is creating the next stage. Be sure your box.json have the properly config to run the test:

box.json

"testbox":{
    "runner":"http://localhost/tests/runner.cfm",
    "reporter":"json",
    "watchers":[ "tests/**.cfc" ],
    "watchDelay":"250"
}

.gitlab-ci.yml

Testbox:
   stage: testbox
   image: docker.toomba.nl/toomba-labs/commandbox:latest
   variables:
      BOX_INSTALL: "true"
   script:
      - box testbox run outputFile='testbox-junit.xml' reporter='junit' > /dev/null
   artifacts:
      paths:
         - testbox-junit.xml
      reports:
         junit: testbox-junit.xml
   only:
      - master

 Mentioned in this post: 

• TestBox

• ColdBox

• Test Driven Development

• CommandBox

• MVC