2.2 Running Tests in Batch Mode

ERT supports automated invocations from the command line or from scripts or makefiles. There are two functions for this purpose, ert-run-tests-batch and ert-run-tests-batch-and-exit. They can be used like this:

emacs -batch -l ert -l my-tests.el -f ert-run-tests-batch-and-exit

This command will start up Emacs in batch mode, load ERT, load my-tests.el, and run all tests defined in it. It will exit with a zero exit status if all tests passed, or nonzero if any tests failed or if anything else went wrong. It will also print progress messages and error diagnostics to standard output.

You can also redirect the above output to a log file, say output.log, and use the ert-summarize-tests-batch-and-exit function to produce a neat summary as shown below:

emacs -batch -l ert -f ert-summarize-tests-batch-and-exit output.log

By default, ERT in batch mode is quite verbose, printing a line with result after each test. This gives you progress information: how many tests have been executed and how many there are. However, in some cases this much output may be undesirable. In this case, set ert-quiet variable to a non-nil value:

emacs -batch -l ert -l my-tests.el \
      --eval "(let ((ert-quiet t)) (ert-run-tests-batch-and-exit))"

In quiet mode ERT prints only unexpected results and summary.

You can specify selectors to only run a subset of your tests (see Test Selectors). For example, the following would run all tests where the name of the test matches the regular expression “to-match”.

emacs -batch -l ert -l my-tests.el \
      -eval '(ert-run-tests-batch-and-exit "to-match")'

By default, ERT test failure summaries are quite brief in batch mode—only the names of the failed tests are listed. If the EMACS_TEST_VERBOSE environment variable is set, the failure summaries will also include the data from the failing test.