19.3 Running testsuite Scripts
Autotest test suites support the following options:
- Display the list of options and exit successfully.
- Display the version of the test suite and exit successfully.
- -C dir
- Change the current directory to dir before creating any files.
Useful for running the testsuite in a subdirectory from a top-level
- Run n tests in parallel, if possible. If n is not given,
run all given tests in parallel. Note that there should be no space
before the argument to -j, as -j number denotes
the separate arguments -j and number, see below.
In parallel mode, the standard input device of the testsuite script is
not available to commands inside a test group. Furthermore, banner
lines are not printed, and the summary line for each test group is
output after the test group completes. Summary lines may appear
unordered. If verbose and trace output are enabled (see below), they
may appear intermixed from concurrently running tests.
Parallel mode requires the mkfifo command to work, and will be
silently disabled otherwise.
- Remove all the files the test suite might have created and exit. Meant
clean Make targets.
- List all the tests (or only the selection), including their possible
By default all tests are performed (or described with --list)
silently in the default environment, but the environment, set of tests,
and verbosity level can be tuned:
- Set the environment variable to value. Use this rather
than ‘FOO=foo ./testsuite’ as debugging scripts would then run in a
AUTOTEST_PATH specifies the testing path to prepend
to PATH. Relative directory names (not starting with
‘/’) are considered to be relative to the top level of the
package being built. All directories are made absolute, first
starting from the top level build tree, then from the
source tree. For instance ‘./testsuite
AUTOTEST_PATH=tests:bin’ for a /src/foo-1.0 source package built
in /tmp/foo results in ‘/tmp/foo/tests:/tmp/foo/bin’ and
then ‘/src/foo-1.0/tests:/src/foo-1.0/bin’ being prepended to
- Add the corresponding test groups, with obvious semantics, to the
- ‘-k keywords’
- Add to the selection the test groups with title or keywords (arguments
AT_KEYWORDS) that match all keywords
of the comma separated list keywords, case-insensitively. Use
‘!’ immediately before the keyword to invert the selection for this
keyword. By default, the keywords match whole words; enclose them in
‘.*’ to also match parts of words.
For example, running
./testsuite -k 'autoupdate,.*FUNC.*'
selects all tests tagged ‘autoupdate’ and with tags
containing ‘FUNC’ (as in ‘AC_CHECK_FUNC’, ‘AC_FUNC_ALLOCA’,
./testsuite -k '!autoupdate' -k '.*FUNC.*'
selects all tests not tagged ‘autoupdate’ or with tags
- If any test fails, immediately abort testing. This implies
--debug: post test group clean up, and top-level logging
are inhibited. This option is meant for the full test
suite, it is not really useful for generated debugging scripts.
If the testsuite is run in parallel mode using --jobs,
then concurrently running tests will finish before exiting.
- Force more verbosity in the detailed output of what is being done. This
is the default for debugging scripts.
- Enable colored test results. Without an argument, or with ‘always’,
test results will be colored. With ‘never’, color mode is turned
off. Otherwise, if either the macro
AT_COLOR_TESTS is used by
the testsuite author, or the argument ‘auto’ is given, then test
results are colored if standard output is connected to a terminal.
- Do not remove the files after a test group was performed—but they are
still removed before, therefore using this option is sane when
running several test groups. Create debugging scripts. Do not
overwrite the top-level
log (in order to preserve a supposedly existing full log file). This is
the default for debugging scripts, but it can also be useful to debug
the testsuite itself.
- Add to the selection all test groups that failed or passed unexpectedly
during the last non-debugging test run.
- Trigger shell tracing of the test groups.
Besides these options accepted by every Autotest testsuite, the
testsuite author might have added package-specific options
(see Writing Testsuites); refer to testsuite --help and
the package documentation for details.