4.7 Unit test modules

A unit test that is a simple C program usually has a module description as simple as this:

     Files:
     tests/test-foo.c
     tests/macros.h
     
     Depends-on:
     
     configure.ac:
     
     Makefile.am:
     TESTS += test-foo
     check_PROGRAMS += test-foo

The test program tests/test-foo.c often has the following structure:

• First comes the obligatory ‘#include <config.h>’.
• Second comes the include of the header file that declares the API being tested. Including it here verifies that said header file is self-contained.
• Then come other includes. In particular, the file macros.h is often used here. It contains a convenient ASSERT macro.

The body of the test, then, contains many ASSERT invocations. When a test fails, the ASSERT macro prints the line number of the failing statement, thus giving you, the developer, an idea of which part of the test failed, even when you don't have access to the machine where the test failed and the reporting user cannot run a debugger.

Sometimes it is convenient to write part of the test as a shell script. (For example, in areas related to process control or interprocess communication, or when different locales should be tried.) In these cases, the typical module description is like this:

     Files:
     tests/test-foo.sh
     tests/test-foo.c
     tests/macros.h
     
     Depends-on:
     
     configure.ac:
     
     Makefile.am:
     TESTS += test-foo.sh
     TESTS_ENVIRONMENT += FOO_BAR='@FOO_BAR@'
     check_PROGRAMS += test-foo

Here, the TESTS_ENVIRONMENT variable can be used to pass values determined by configure or by the Makefile to the shell script, as environment variables. The values of EXEEXT and of srcdir, from Autoconf and Automake, are already provided as environment variables, through an initial value of TESTS_ENVIRONMENT that gnulib-tool puts in place.

Regardless of the specific form of the unit test, the following guidelines should be respected:

• A test indicates success by exiting with exit code 0. It should normally not produce output in this case. (Output to temporary files that are cleaned up at the end of the test are possible, of course.)
• A test indicates failure by exiting with an exit code different from 0 and 77, typically 1. It is useful to print a message about the failure in this case. The ASSERT macro already does so.
• A test indicates "skip", that is, that most of its interesting functionality could not be performed, through a return code of 77. A test should also print a message to stdout or stderr about the reason for skipping. For example:

              fputs ("Skipping test: multithreading not enabled\n", stderr);
              return 77;
  

  Such a message helps detecting bugs in the autoconf macros: A simple message ‘SKIP: test-foo’ does not sufficiently catch the attention of the user.