Contexts

Contexts are a facility provided by the Unit Test Framework in order to be able to trace the location of assertions better. To grasp the idea, consider the following example:

void test_operations(Processor& processor, int limit)
{
  for (int i = 0; i < limit; ++i) {
    BOOST_TEST(processor.op1(i));
    for (int j = 0; j < i; ++j) {
      BOOST_TEST(processor.op2(i, j));
    }
  }
}

In case of failure, in order to see in the logs at which point of the loops the failure occurred, we need some extra information in the assertion, which can be achieved for instance the following way:

BOOST_TEST(processor.op1(i));

replaced by

BOOST_TEST(processor.op1(i), "With parameter i = " << i);

We see in this trivial example that a context, which is the variable i in this case, should be acknowledged by the assertion BOOST_CHECK in a particular way. In the approach above, this is done by adding a message to the assertion itself.

What if the context is more complex than that? In case the complexity of the context increases, the fact that the assertion and the context is tightly coupled as in the approach above is difficult to maintain:

void test_operations(Processor& processor, int limit, int level)
{
  for (int i = 0; i < limit; ++i) {
    BOOST_TEST(processor.op1(i),
               "With optimization level " << level << ", With parameter i = " << i);
    for (int j = 0; j < i; ++j) {
      BOOST_TEST(processor.op2(i, j),
                 "With optimization level " << level <<
                 ", With parameter i = " << i << ", With parameter j = " << j);
    }
  }
}

BOOST_AUTO_TEST_CASE(test1)
{
  Processor processor;

  for (int level = 0; level < 3; ++level) {
    processor.optimization_level(level);
    test_operations(processor, 2, level);
  }
}

Note the length of the messages, the repetition, and the fact, that we pass argument level to function test_operations only for the sake of generating an error message in case of a failure.

Therefore, loose coupling between the context of an assertion and the assertion point is a property that is desirable.

Assertion-bound context

BOOST_TEST_INFO can be used to define an error message to be bound to the first following assertion. If (and only if) the assertion fails, the bound message will be displayed along:

Example: Assertion-bound context

Code
#define BOOST_TEST_MODULE example80 #include <boost/test/included/unit_test.hpp> void test() { BOOST_TEST(false); } BOOST_AUTO_TEST_CASE(test_case1) { BOOST_TEST_INFO("Alpha"); BOOST_TEST_INFO("Beta"); BOOST_TEST(true); BOOST_TEST_INFO("Gamma"); char a = 'a'; BOOST_TEST_INFO("Delt" << a); test(); }

Code

#define BOOST_TEST_MODULE example80
#include <boost/test/included/unit_test.hpp>

void test()
{
  BOOST_TEST(false);
}

BOOST_AUTO_TEST_CASE(test_case1)
{
  BOOST_TEST_INFO("Alpha");
  BOOST_TEST_INFO("Beta");
  BOOST_TEST(true);

  BOOST_TEST_INFO("Gamma");
  char a = 'a';
  BOOST_TEST_INFO("Delt" << a);
  test();
}

Output
> example Running 1 test case... test.cpp(14): error: in "test_case1": check false has failed Failure occurred in a following context: Gamma Delta *** 1 failures is detected in test module "example80"

The information composed inside BOOST_TEST_INFO is bound only to the first assertion following the declaration. This information is only displayed if the assertion fails; otherwise the message is discarded. The BOOST_TEST_INFO declaration does not have to immediately precede the assertion, it is allowed to intertwine them with other instructions, they can even be declared in different scopes. It is also possible to bind more than one information to a given assertion.

With BOOST_TEST_INFO, we can improve our initial example as follows:

void test_operations(Processor& processor, int limit, int level)
{
  for (int i = 0; i < limit; ++i) {
    BOOST_TEST_INFO("With optimization level " << level);
    BOOST_TEST_INFO("With parameter i = " << i);
    BOOST_TEST(processor.op1(i));
    for (int j = 0; j < i; ++j) {
      BOOST_TEST_INFO("With optimization level " << level);
      BOOST_TEST_INFO("With parameter i = " << i);
      BOOST_TEST_INFO("With parameter j = " << j);
      BOOST_TEST(processor.op2(i, j));
    }
  }
}

BOOST_AUTO_TEST_CASE(test1)
{
  Processor processor;

  for (int level = 0; level < 3; ++level) {
    processor.optimization_level(level);
    test_operations(processor, 2, level);
  }
}

Scope-bound context

In the previous example, the information stored inside the calls to BOOST_TEST_INFO were all consumed by the next assertion. There are cases where we would like this information be persistent for the current scope. Unit Test Framework provides two tools to achieve this:

BOOST_TEST_CONTEXT defines a diagnostic message and a scope. The message is bound to every assertion in that scope, and is displayed along with every failed assertion.
BOOST_TEST_INFO_SCOPE acts the same as BOOST_TEST_INFO, but the stored context information is bound to all the assertions that follow the call to BOOST_TEST_INFO_SCOPE within the current scope.

	Tip
	Since Boost Boost 1.70, `BOOST_TEST_CONTEXT` can accept multiple arguments.

	Tip
	`BOOST_TEST_INFO_SCOPE` has been introduced in Boost 1.70.

Example: Scope-bound context

Code
#define BOOST_TEST_MODULE example81 #include <boost/test/included/unit_test.hpp> void test() { BOOST_TEST(2 != 2); } BOOST_AUTO_TEST_CASE(test_case1) { BOOST_TEST_CONTEXT("Alpha") { BOOST_TEST(1 != 1); test(); BOOST_TEST_CONTEXT("Be" << "ta") BOOST_TEST(3 != 3); BOOST_TEST(4 == 4); } BOOST_TEST(5 != 5); }

Code

#define BOOST_TEST_MODULE example81
#include <boost/test/included/unit_test.hpp>

void test()
{
  BOOST_TEST(2 != 2);
}

BOOST_AUTO_TEST_CASE(test_case1)
{
  BOOST_TEST_CONTEXT("Alpha") {
    BOOST_TEST(1 != 1);
    test();

    BOOST_TEST_CONTEXT("Be" << "ta")
      BOOST_TEST(3 != 3);

    BOOST_TEST(4 == 4);
  }

  BOOST_TEST(5 != 5);
}

Output
> example Running 1 test case... test.cpp(20): error: in "test_case1": check 1 != 1 has failed [1 == 1] Failure occurred in a following context: Alpha test.cpp(14): error: in "test_case1": check 2 != 2 has failed [2 == 2] Failure occurred in a following context: Alpha test.cpp(24): error: in "test_case1": check 3 != 3 has failed [3 == 3] Failure occurred in a following context: Alpha Beta test.cpp(29): error: in "test_case1": check 5 != 5 has failed [5 == 5] *** 4 failures are detected in test module "example81"

Output

> example
Running 1 test case...
test.cpp(20): error: in "test_case1": check 1 != 1 has failed [1 == 1]
Failure occurred in a following context:
    Alpha
test.cpp(14): error: in "test_case1": check 2 != 2 has failed [2 == 2]
Failure occurred in a following context:
    Alpha
test.cpp(24): error: in "test_case1": check 3 != 3 has failed [3 == 3]
Failure occurred in a following context:
    Alpha
    Beta
test.cpp(29): error: in "test_case1": check 5 != 5 has failed [5 == 5]

*** 4 failures are detected in test module "example81"

In the previous example, there is an opening brace right after BOOST_TEST_CONTEXT: this pair of braces defines the scope in which the diagnostic message is in effect. If there is no braces, the scope applies only to the following statement. BOOST_TEST_CONTEXT declarations can nest.

With BOOST_TEST_CONTEXT, we can further improve our initial example, by putting variable level into a scope-level context and not pass it as function parameter:

void test_operations(Processor& processor, int limit)
{
  for (int i = 0; i < limit; ++i) {
    BOOST_TEST_INFO("With parameter i = " << i);
    BOOST_TEST(processor.op1(i));
    for (int j = 0; j < i; ++j) {
      BOOST_TEST_INFO("With parameter i = " << i);
      BOOST_TEST_INFO("With parameter j = " << j);
      BOOST_TEST(processor.op2(i, j));
    }
  }
}

BOOST_AUTO_TEST_CASE(test1)
{
  Processor processor;

  for (int level = 0; level < 3; ++level) {
    BOOST_TEST_CONTEXT("With optimization level " << level) {
      processor.optimization_level(level);
      test_operations(processor, 2);
    }
  }
}

If we observe that variable i also applies in a certain scope, we can improve our example further still.

Example: Using contexts

Code
#define BOOST_TEST_MODULE example82 #include <boost/test/included/unit_test.hpp> struct Processor { int level; void optimization_level(int l) { level = l; } bool op1(int) { return level < 2; } bool op2(int, int) { return level < 1; } }; void test_operations(Processor& processor, int limit) { for (int i = 0; i < limit; ++i) { BOOST_TEST_CONTEXT("With parameter i = " << i) { BOOST_TEST(processor.op1(i)); for (int j = 0; j < i; ++j) { BOOST_TEST_INFO("With parameter j = " << j); BOOST_TEST(processor.op2(i, j)); } } } } BOOST_AUTO_TEST_CASE(test1) { Processor processor; for (int level = 0; level < 3; ++level) { BOOST_TEST_CONTEXT("With optimization level " << level) { processor.optimization_level(level); test_operations(processor, 2); } } }

Code

#define BOOST_TEST_MODULE example82
#include <boost/test/included/unit_test.hpp>

struct Processor
{
  int level;
  void optimization_level(int l) { level = l; }
  bool op1(int) { return level < 2; }
  bool op2(int, int) { return level < 1; }
};

void test_operations(Processor& processor, int limit)
{
  for (int i = 0; i < limit; ++i) {
    BOOST_TEST_CONTEXT("With parameter i = " << i) {
      BOOST_TEST(processor.op1(i));
      for (int j = 0; j < i; ++j) {
        BOOST_TEST_INFO("With parameter j = " << j);
        BOOST_TEST(processor.op2(i, j));
      }
    }
  }
}

BOOST_AUTO_TEST_CASE(test1)
{
  Processor processor;

  for (int level = 0; level < 3; ++level) {
    BOOST_TEST_CONTEXT("With optimization level " << level) {
      processor.optimization_level(level);
      test_operations(processor, 2);
    }
  }
}

Output
> example Running 1 test case... test.cpp(27): error: in "test1": check processor.op2(i, j) has failed Failure occurred in a following context: With optimization level 1 With parameter i = 1 With parameter j = 0 test.cpp(24): error: in "test1": check processor.op1(i) has failed Failure occurred in a following context: With optimization level 2 With parameter i = 0 test.cpp(24): error: in "test1": check processor.op1(i) has failed Failure occurred in a following context: With optimization level 2 With parameter i = 1 test.cpp(27): error: in "test1": check processor.op2(i, j) has failed Failure occurred in a following context: With optimization level 2 With parameter i = 1 With parameter j = 0 *** 4 failures are detected in the test module "example82"

Output

> example
Running 1 test case...
test.cpp(27): error: in "test1": check processor.op2(i, j) has failed
Failure occurred in a following context:
    With optimization level 1
    With parameter i = 1
    With parameter j = 0
test.cpp(24): error: in "test1": check processor.op1(i) has failed
Failure occurred in a following context:
    With optimization level 2
    With parameter i = 0
test.cpp(24): error: in "test1": check processor.op1(i) has failed
Failure occurred in a following context:
    With optimization level 2
    With parameter i = 1
test.cpp(27): error: in "test1": check processor.op2(i, j) has failed
Failure occurred in a following context:
    With optimization level 2
    With parameter i = 1
    With parameter j = 0

*** 4 failures are detected in the test module "example82"

Finally, it is possible to pass several arguments to BOOST_TEST_CONTEXT, which is more convenient than having several scopes:

Example: Multiple arguments to `BOOST_TEST_CONTEXT`

Code
#define BOOST_TEST_MODULE example83 multicontext #include <boost/test/included/unit_test.hpp> #include <cmath> BOOST_AUTO_TEST_CASE(test_multi_context) { for (int level = 0; level < 10; ++level) { int rand_value = std::abs(rand()) % 50; BOOST_TEST_CONTEXT("With level " << level, "Random value=" << rand_value){ for( int j = 1; j < rand_value; j++) { BOOST_TEST(level < rand_value); rand_value /= 2; } } } }

Code

#define BOOST_TEST_MODULE example83 multicontext
#include <boost/test/included/unit_test.hpp>
#include <cmath>

BOOST_AUTO_TEST_CASE(test_multi_context)
{
  for (int level = 0; level < 10; ++level) {
    int rand_value = std::abs(rand()) % 50;
    BOOST_TEST_CONTEXT("With level " << level, "Random value=" << rand_value){
      for( int j = 1; j < rand_value; j++) {
        BOOST_TEST(level < rand_value);
        rand_value /= 2;
      }
    }
  }
}

Output
> example Running 1 test case... test.cpp(19): error: in "test_multi_context": check level < rand_value has failed [7 >= 5] Failure occurred in a following context: With level 7 Random value=42 test.cpp(19): error: in "test_multi_context": check level < rand_value has failed [8 >= 6] Failure occurred in a following context: With level 8 Random value=49 test.cpp(19): error: in "test_multi_context": check level < rand_value has failed [9 >= 5] Failure occurred in a following context: With level 9 Random value=21 *** 3 failures are detected in the test module "example83 multicontext"

Output

> example
Running 1 test case...
test.cpp(19): error: in "test_multi_context": check level < rand_value has failed [7 >= 5]
Failure occurred in a following context:
    With level 7
    Random value=42
test.cpp(19): error: in "test_multi_context": check level < rand_value has failed [8 >= 6]
Failure occurred in a following context:
    With level 8
    Random value=49
test.cpp(19): error: in "test_multi_context": check level < rand_value has failed [9 >= 5]
Failure occurred in a following context:
    With level 9
    Random value=21

*** 3 failures are detected in the test module "example83 multicontext"

BOOST_TEST_INFO_SCOPE is convenient when you augment the current scope information as new information arrives. The following example calls several time a quadratic polynomial estimation function with random polynomial. As the random values are drawn in a loop, they are placed in the current scope with BOOST_TEST_INFO_SCOPE, which allows us to easily debug the function.

Example: Sticky context with `BOOST_TEST_INFO_SCOPE`

Code
#define BOOST_TEST_MODULE example84 #include <boost/test/included/unit_test.hpp> #include <random> #include <cmath> // this function does not compute properly the polynomial root estimation // in the case of a double root. template <class random_generator_t> std::pair<double, double> estimate_polynomial_roots( random_generator_t& gen, std::function<double(double)> polynomial) { using namespace std; std::uniform_real_distribution<> dis(-10, 10); double x1 = dis(gen); double x2 = dis(gen); double fx1 = polynomial(x1); double fx2 = polynomial(x2); BOOST_TEST_INFO_SCOPE("sample1 = " << x1); BOOST_TEST_INFO_SCOPE("sample2 = " << x2); // from Vieta formula double minus_b = x2 + x1 - (fx2 - fx1) / (x2 - x1); double c = (x1 * fx2 - x2 * fx1 + x2 * x1 * x1 - x1 * x2 * x2) / (x1 - x2); BOOST_TEST(minus_b * minus_b >= 4c); return std::make_pair( (minus_b - sqrt(minus_b minus_b - 4 * c)) / 2, (minus_b + sqrt(minus_b * minus_b - 4 * c)) / 2); } BOOST_AUTO_TEST_CASE(quadratic_estimation) { std::random_device rd; unsigned int seed = rd(); std::mt19937 gen(seed); std::uniform_int_distribution<> dis(-10, 10); BOOST_TEST_MESSAGE("Seed = " << seed); for(int i = 0; i < 50; i++) { BOOST_TEST_INFO_SCOPE("trial " << i+1); int root1 = dis(gen); int root2 = dis(gen); if(root1 > root2) { std::swap(root1, root2); } BOOST_TEST_INFO_SCOPE("root1 = " << root1); BOOST_TEST_INFO_SCOPE("root2 = " << root2); std::pair<double, double> estimated = estimate_polynomial_roots( gen, [root1, root2](double x) -> double { return (x - root1) * (x - root2); }); BOOST_TEST(estimated.first == double(root1), 10. % boost::test_tools::tolerance()); BOOST_TEST(estimated.second == double(root2), 10. % boost::test_tools::tolerance()); } }

Code

#define BOOST_TEST_MODULE example84
#include <boost/test/included/unit_test.hpp>
#include <random>
#include <cmath>

// this function does not compute properly the polynomial root estimation
// in the case of a double root.
template <class random_generator_t>
std::pair<double, double> estimate_polynomial_roots(
  random_generator_t& gen,
  std::function<double(double)> polynomial) {

  using namespace std;

  std::uniform_real_distribution<> dis(-10, 10);
  double x1 = dis(gen);
  double x2 = dis(gen);
  double fx1 = polynomial(x1);
  double fx2 = polynomial(x2);

  BOOST_TEST_INFO_SCOPE("sample1 = " << x1);
  BOOST_TEST_INFO_SCOPE("sample2 = " << x2);

  // from Vieta formula
  double minus_b = x2 + x1 - (fx2 - fx1) / (x2 - x1);
  double c = (x1 * fx2 - x2 * fx1 + x2 * x1 * x1 - x1 * x2 * x2) / (x1 - x2);

  BOOST_TEST(minus_b * minus_b >= 4*c);

  return std::make_pair(
    (minus_b - sqrt(minus_b * minus_b - 4 * c)) / 2,
    (minus_b + sqrt(minus_b * minus_b - 4 * c)) / 2);
}

BOOST_AUTO_TEST_CASE(quadratic_estimation)
{
  std::random_device rd;
  unsigned int seed = rd();
  std::mt19937 gen(seed);
  std::uniform_int_distribution<> dis(-10, 10);

  BOOST_TEST_MESSAGE("Seed = " << seed);

  for(int i = 0; i < 50; i++) {
    BOOST_TEST_INFO_SCOPE("trial " << i+1);
    int root1 = dis(gen);
    int root2 = dis(gen);
    if(root1 > root2) {
      std::swap(root1, root2);
    }
    BOOST_TEST_INFO_SCOPE("root1 = " << root1);
    BOOST_TEST_INFO_SCOPE("root2 = " << root2);

    std::pair<double, double> estimated = estimate_polynomial_roots(
      gen,
      [root1, root2](double x) -> double { return (x - root1) * (x - root2); });

    BOOST_TEST(estimated.first == double(root1), 10. % boost::test_tools::tolerance());
    BOOST_TEST(estimated.second == double(root2), 10. % boost::test_tools::tolerance());
  }
}

Output
> example --log_level=message Seed = 162981956 test.cpp(34): error: in "quadratic_estimation": check minus_b * minus_b >= 4c has failed [-13.999999999999998 -13.999999999999998 < 195.99999999999997] Failure occurred in a following context: trial 14 root1 = -7 root2 = -7 sample1 = 4.66289 sample2 = 1.70234 test.cpp(64): error: in "quadratic_estimation": check estimated.first == double(root1) has failed [-nan != -7]. Relative difference exceeds tolerance [-nan > 0.1] Failure occurred in a following context: trial 14 root1 = -7 root2 = -7 test.cpp(65): error: in "quadratic_estimation": check estimated.second == double(root2) has failed [-nan != -7]. Relative difference exceeds tolerance [-nan > 0.1] Failure occurred in a following context: trial 14 root1 = -7 root2 = -7 *** 3 failures are detected in the test module "example84"

Output

> example --log_level=message
Seed = 162981956
test.cpp(34): error: in "quadratic_estimation": check minus_b * minus_b >= 4*c has failed [-13.999999999999998 * -13.999999999999998 < 195.99999999999997]
Failure occurred in a following context:
    trial 14
    root1 = -7
    root2 = -7
    sample1 = 4.66289
    sample2 = 1.70234
test.cpp(64): error: in "quadratic_estimation": check estimated.first == double(root1) has failed [-nan != -7]. Relative difference exceeds tolerance [-nan > 0.1]
Failure occurred in a following context:
    trial 14
    root1 = -7
    root2 = -7
test.cpp(65): error: in "quadratic_estimation": check estimated.second == double(root2) has failed [-nan != -7]. Relative difference exceeds tolerance [-nan > 0.1]
Failure occurred in a following context:
    trial 14
    root1 = -7
    root2 = -7

*** 3 failures are detected in the test module "example84"