...one of the most highly
regarded and expertly designed C++ library projects in the
world.
— Herb Sutter and Andrei
Alexandrescu, C++
Coding Standards
If you consider test cases as leaves on the test tree, the test suite can be considered as branch and the master test suite as the root. Unlike real trees though, our tree in many cases consists only of leaves attached directly to the root. This is common for all test cases to reside directly in the master test suite. If you do want to construct a hierarchical test suite structure the Unit Test Framework provides both manual and automated test suite creation and registration facilities:
In addition the Unit Test Framework presents a notion of the Master test suite. The most important reason to learn about this component is that it provides an ability to access command line arguments supplied to a test module.
The solution the Unit Test Framework presents for automated test suite creation and registration is designed to facilitate multiple points of definition, arbitrary test suites depth and smooth integration with automated test case creation and registration. This facility should significantly simplify a test tree construction process in comparison with manual explicit registration case.
The implementation is based on the order of file scope variables definitions
within a single compilation unit. The semantic of this facility is very
similar to the namespace feature of C++, including support for test suite
extension. To start test suite use the macro BOOST_AUTO_TEST_SUITE
. To end
test suite use the macro BOOST_AUTO_TEST_SUITE_END
. The
same test suite can be restarted multiple times inside the same test file
or in a different test files. In a result all test units will be part of
the same test suite in a constructed test tree.
BOOST_AUTO_TEST_SUITE(test_suite_name); BOOST_AUTO_TEST_SUITE_END();
Test units defined in between test suite start and end declarations become members of the test suite. A test unit always becomes the member of the closest test suite declared. Test units declared at a test file scope become members of the master test suite. There is no limit on depth of test suite inclusion.
This example creates a test tree that matches exactly the one created in the manual test suite registration example.
Code |
---|
#define BOOST_TEST_MODULE example #include <boost/test/included/unit_test.hpp> BOOST_AUTO_TEST_SUITE( test_suite1 ) BOOST_AUTO_TEST_CASE( test_case1 ) { BOOST_TEST_WARN( sizeof(int) < 4U ); } BOOST_AUTO_TEST_CASE( test_case2 ) { BOOST_TEST_REQUIRE( 1 == 2 ); BOOST_FAIL( "Should never reach this line" ); } BOOST_AUTO_TEST_SUITE_END() BOOST_AUTO_TEST_SUITE( test_suite2 ) BOOST_AUTO_TEST_CASE( test_case3 ) { BOOST_TEST( true ); } BOOST_AUTO_TEST_CASE( test_case4 ) { BOOST_TEST( false ); } BOOST_AUTO_TEST_SUITE_END() |
Output |
---|
> example Running 4 test cases... test.cpp(21): fatal error: in "test_suite1/test_case2": critical check 1 == 2 has failed [1 != 2] test.cpp(35): error: in "test_suite2/test_case4": check false has failed *** 2 failures are detected in the test module "example" |
As you can see test tree construction in this example is more straightforward and automated.
In the example below, the test suite test_suite
consists of two parts. Their definition is remote and is separated by another
test case. In fact these parts may even reside in different test files.
The resulting test tree remains the same. As you can see from the output
both test_case1
and test_case2
reside in the same test suite
test_suite
.
Code |
---|
#define BOOST_TEST_MODULE example #include <boost/test/included/unit_test.hpp> BOOST_AUTO_TEST_SUITE( test_suite ) BOOST_AUTO_TEST_CASE( test_case1 ) { BOOST_ERROR( "some error 1" ); } BOOST_AUTO_TEST_SUITE_END() BOOST_AUTO_TEST_CASE( test_case_on_file_scope ) { BOOST_TEST( true ); } BOOST_AUTO_TEST_SUITE( test_suite ) BOOST_AUTO_TEST_CASE( test_case2 ) { BOOST_ERROR( "some error 2" ); } BOOST_AUTO_TEST_SUITE_END() |
Output |
---|
>example --report_level=detailed Running 3 test cases... test.cpp(8): error in "test_case1": some error 1 test.cpp(23): error in "test_case2": some error 2 Test suite "example" failed with: 1 assertion out of 3 passed 2 assertions out of 3 failed 1 test case out of 3 passed 2 test cases out of 3 failed Test suite "test_suite" failed with: 2 assertions out of 2 failed 2 test cases out of 2 failed Test case "test_case1" failed with: 1 assertion out of 1 failed Test case "test_case2" failed with: 1 assertion out of 1 failed Test case "test_case_on_file_scope" passed with: 1 assertion out of 1 passed |
To create a test suite manually you need to
boost::unit_test::test_suite
class,
The Unit Test Framework models the notion of test
case container - test suite - using class boost::unit_test::test_suite
.
For complete class interface reference check advanced section of this documentation.
Here you should only be interested in a single test unit registration interface:
void test_suite::add( test_unit* tc, counter_t expected_failures = 0, int timeout = 0 );
The first parameter is a pointer to a newly created test unit. The second optional parameter - expected_failures - defines the number of test assertions that are expected to fail within the test unit. By default no errors are expected.
Caution | |
---|---|
Be careful when supplying a number of expected failures for test suites. By default the Unit Test Framework calculates the number of expected failures in test suite as the sum of appropriate values in all test units that constitute it. And it rarely makes sense to change this. |
The third optional parameter - timeout
- defines the timeout value for the test unit. As of now the Unit
Test Framework isn't able to set a timeout for the test suite
execution, so this parameter makes sense only for test case registration.
By default no timeout is set. See the method boost::unit_test::test_suite::add
for more details about the timeout value.
To register group of test units in one function call, the test_suite
class provides
another add
interface covered in
the advanced section of this documentation.
To create a test suite instance manually, employ the macro BOOST_TEST_SUITE
. It hides all
implementation details and you only required to specify the test suite
name:
BOOST_TEST_SUITE(test_suite_name);
BOOST_TEST_SUITE
creates an instance
of the class boost::unit_test::test_suite
and returns a pointer to the
constructed instance. Alternatively you can create an instance of class
boost::unit_test::test_suite
yourself.
Caution | |
---|---|
|
Newly created test suite has to be registered in a parent one using the
add
interface. Both test
suite creation and registration is performed in the test module initialization
function.
The example below creates a test tree, which can be represented by the following hierarchy:
Code |
---|
#include <boost/test/included/unit_test.hpp> using namespace boost::unit_test; void test_case1() { /* ... */ } void test_case2() { /* ... */ } void test_case3() { /* ... */ } void test_case4() { /* ... */ } test_suite* init_unit_test_suite( int /*argc*/, char* /*argv*/[] ) { test_suite* ts1 = BOOST_TEST_SUITE( "test_suite1" ); ts1->add( BOOST_TEST_CASE( &test_case1 ) ); ts1->add( BOOST_TEST_CASE( &test_case2 ) ); test_suite* ts2 = BOOST_TEST_SUITE( "test_suite2" ); ts2->add( BOOST_TEST_CASE( &test_case3 ) ); ts2->add( BOOST_TEST_CASE( &test_case4 ) ); framework::master_test_suite().add( ts1 ); framework::master_test_suite().add( ts2 ); return 0; } |
Output |
---|
> example --log_level=test_suite Running 4 test cases... Entering test suite "Master Test Suite" Entering test suite "test_suite1" Entering test case "test_case1" Leaving test case "test_case1" Entering test case "test_case2" Leaving test case "test_case2" Leaving test suite "test_suite1" Entering test suite "test_suite2" Entering test case "test_case3" Leaving test case "test_case3" Entering test case "test_case4" Leaving test case "test_case4" Leaving test suite "test_suite2" Leaving test suite "Master Test Suite" *** No errors detected |