.. _test-reporter:
.. index:: ! test reporters

Test reporters
==============

Test reporters are responsible for the output from a test
applicaton. It is up to the reporter to decide the format of the
output. This could be xml, json or even messages to some other
application, e.g. a GUI application for running tests. Currently there
is a text reporter implemented.

Test reporter may be loaded into the test runner either at
construction or by calling the :func:`test_runner::use_reporter()`
method.

Don't let more than one test runner reference an instance of a test
reporter at a time. If several test runners use the same reporter the
output will most likely be interleaved in an unfortunate way.

The ``test_reporter`` interface
-------------------------------

It is a fairly easy task to implement new test reporters. Each test
reporter must be derived from the ``test_reporter`` base class.

.. class:: test_reporter

   .. function:: test_reporter()

   .. function:: ~test_reporter()

      ::

        virtual ~test_reporter()

   .. function:: prepare_run()
   
      ::

        virtual void prepare_run(const run_descriptor &d)

      This function is called once before any tests are run.    

   .. function:: prepare_suite()
       
      ::

        virtual void prepare_suite(long suiteId);

      This function is called once before any tests in a given suite
      are run. The ``suiteId`` argument is a generated id that
      uniquely identifies the test suite.

   .. function:: prepare_test()
       
      ::

        virtual void prepare_test(long testId)

      This function is called before each test. The ``testId``
      argument is a generated id that uniquely identifies the test.

   .. function:: complete_test()
       
      ::

        virtual void complete_test(const run_result &r)

      This function is called after each test.    

   .. function:: complete_suite()
       
      ::

        virtual void complete_suite(long suiteId)

      This function is called after the last test in the suite has been run.  

   .. function:: complete_run()
       
      ::

        virtual void complete_run()

      This function is called when all tests have been run.  

   .. function:: run_error()
       
      ::

        virtual void run_error(const run_result &r)

      This function is called if some kind of error was detected.

It is up to the reporter to decide what to do on each function
call. The ``run_error()`` method will be called if some error is
detected during the execution. E.g it may be due to unexpected
exceptions during setup or teardown. However, if any of the
``prepare_*()`` methods have been called, the corresponding
``complete_*()`` method is guaranteed to be called after a
``run_error()``. Hence, suppose that an error occures after the first
``prepare_suite()`` method has been called. Then the call sequence will
be ::
  
  prepare_run(...)
  prepare_suite(...)
  run_error(...)
  complete_suite(...)
  // ... 
  // more calls for other suites
  // ...
  complete_run(...)


.. index:: text_reporter

The ``text_reporter``
---------------------

The default reporter is the ``text_reporter``. It will print out the
result of a test run to an ``ostream``, the default is ``cout``. To
use another stream, e.g. a file stream, construct a new
``text_reporter`` with the stream as an argument and load it into the
test runner. 

.. class:: text_reporter

   .. function:: text_reporter()

      ::

        text_reporter(std::ostream &output = std::cout, mode m = DETAILED)

      Constructs a text reporter that will output the result to stream
      ``output`` using mode ``m``. See the :type:`text_reporter::mode`
      for more details.

   .. member:: mode Mode

      Gets or sets the output mode of the reporter.

   .. type:: mode

      The mode enumeration type is used in order to set how detailed
      the output from the reporter is to be. Possible values are

      QUIET
        No output.

      DETAILED
        Maximum output. This mode will print the name of each suite,
        followed by the description of each test. It will indicate
        which tests that fails and which ones that passes. This mode
        will also print a suite summary and a total summary. Fail and
        error reports will be printed after information about all
        suites.

      SUITE_SUMMARY 
        This mode will print the description of each suite followed by
        a summary how many tests that were run, how many that passed,
        and how many that failed in the suite.

      TOTAL_SUMMARY
        This mode will only print a summary indicating how many suites
        that were run, how many tests that were run, and how many of
        them that passed and failed.

::

  // Create a text_reporter that outputs a suite summaries to a file.
  std::fstream MyFile("my_report.txt");
  text_reporter MyReporter(MyFile, text_reporter::SUITE_SUMMARY);

  // If test runner has not been created yet...
  test_runner Tester(MyReporter);

  // or if the test runner is already defined
  Tester.use_reporter(MyReporter);


.. index:: multi_reporter

The ``multi_reporter``
----------------------

The ``multi_reporter`` is simply a relay for calls to test_reporter
methods. It will make each call to a list of test reporters. The order
in which the reporters are called is the same as the order in which
they were added to the ``multi_reporter``.

.. class:: multi_reporter

   .. function:: multi_reporter()

   .. function:: add()

      ::

          void add(test_reporter *)

      Add a new ``test_reporter`` to the list of test reporters to
      call. The ``multi_reporter`` will grab ownership of the added
      reporter.

::

  // Write test results to two different reporters, a detailed 
  // report to a file, and a summary to the console.

  std::fstream MyFile("my_report.txt");

  multi_reporter reporter;
  reporter.add(new text_reporter(MyFile, text_reporter::DETAILED));
  reporter.add(new text_reporter(cout, text_reporter::SUITE_SUMMARY));

.. index:: run_descriptor, suite_descriptior, test_descriptor

The ``run_descriptor`` class
----------------------------

The ``run_descriptor`` class is a data structure that describes the
test run to be performed. It contains a list of instances of the
``suite_descriptor`` class. Each ``suite_descriptor`` contains a list
of ``test_descriptor`` instances. Hence a run descriptor describes a
test hierarchy.


.. index:: <<, operator<<
    
Suppress usage of ``<<`` operator
---------------------------------

The ``verify`` function and different constraints will use the ``<<``
operator in order to print values of arguments in test reports if some
test fails. If the operator is not defined for some class or type this
will result in a compile time error. The obvious, and a bit clumsy way
around this, is to define the ``<<`` operator for the type. This may
be tedious to do in some meaningful way. There is another way however,
by specializing the ``unittest_traits`` class for the type. ::

  template<>
  class unittest_traits<some_non_printable_class> {
  public:
    typedef some_non_printable_class   value_type;
    typedef not_printable_tag          print_category;
  };

If a type has the ``not_printable_tag`` set, the type will not be
output via the ``<<`` operator, instead the string "not printable"
will be used.