Boost C++ Libraries

In general, it is a good practice to write any test as independent as possible from any other, there are however cases where a dependency cannot be avoided and an order for executing the tests is needed.

In the general setup and for any two test cases TA and TB, TB should not take for granted that TA has already executed, even if TA is declared before TB in the same translation unit. The only ordering-related guarantee that Unit Test Framework makes by default is that if test cases TA and TB are declared in the same test suite, no test case (call it TX) from any other test suite is executed between TA and TB, even if the declaration of TX appears between the declarations of TA and TB. In other words, all tests from a suite are executed in one go, even if the test suite namespace is opened multiple times.

Even though the order is not guaranteed, it may accidentally be preserved across the different runs. In order to make sure the test cases do not depend on one another, the test module may be called with an additional command-line argument, random, to shuffle the tests unit ordering and to be more robust against an erroneous implicit ordering.

Declaring a test case dependency

If there exist a dependency between test units, and an ordering is required between the execution of those tests, it has to be declared explicitly. Dependencies in the Unit Test Framework affect two dimensions of test units, which are:

Decorator depends_on associates the decorated test case (call it TB) with another test case (call it TA) specified by name. This affects the processing the test tree in two ways:

Example: decorator depends_on

#define BOOST_TEST_MODULE decorator_07
#include <boost/test/included/unit_test.hpp>

namespace utf = boost::unit_test;

// test1 and test2 defined at the bottom

BOOST_AUTO_TEST_CASE(test3, * utf::depends_on("s1/test1"))
{
  BOOST_TEST(false);
}

BOOST_AUTO_TEST_CASE(test4, * utf::depends_on("test3"))
{
  BOOST_TEST(false);
}

BOOST_AUTO_TEST_CASE(test5, * utf::depends_on("s1/test2"))
{
  BOOST_TEST(false);
}

BOOST_AUTO_TEST_SUITE(s1)

  BOOST_AUTO_TEST_CASE(test1)
  {
    BOOST_TEST(true);
  }

  BOOST_AUTO_TEST_CASE(test2, * utf::disabled())
  {
    BOOST_TEST(false);
  }

BOOST_AUTO_TEST_SUITE_END()

> decorator_07 --report_level=detailed --log_level=all
Running 4 test cases...
Entering test module "decorator_07"
test.cpp:31: Entering test suite "s1"
test.cpp:33: Entering test case "test1"
test.cpp:35: info: check true has passed
test.cpp:33: Leaving test case "test1"; testing time: 100us
test.cpp:31: Leaving test suite "s1"; testing time: 129us
test.cpp:16: Entering test case "test3"
test.cpp:18: error: in "test3": check false has failed
test.cpp:16: Leaving test case "test3"; testing time: 48us
test.cpp:26: Test case "test5" is skipped because dependency test case "s1/test2" is disabled
test.cpp:21: Test case "test4" is skipped because dependency test case "test3" has failed
Leaving test module "decorator_07"; testing time: 263us

Test module "decorator_07" has failed with:
  1 test case out of 4 passed
  1 test case out of 4 failed
  2 test cases out of 4 skipped
  1 assertion out of 2 passed
  1 assertion out of 2 failed

  Test case "test3" has failed with:
    1 assertion out of 1 failed

  Test case "test4" was skipped
  Test case "test5" was skipped
  Test suite "s1" has passed with:
    1 test case out of 1 passed
    1 assertion out of 1 passed

    Test case "s1/test1" has passed with:
      1 assertion out of 1 passed

In the above scenario: