Sometimes, it doesn’t make sense to run a test due to missing
preconditions. A required Emacs feature might not be compiled in, the
function to be tested could call an external binary which might not be
available on the test machine, you name it. In this case, the macro
skip-unless could be used to skip the test:
(ert-deftest test-dbus () "A test that checks D-BUS functionality." (skip-unless (featurep 'dbusbind)) ...)
The outcome of running a test should not depend on the current state of the environment, and each test should leave its environment in the same state it found it in. In particular, a test should not depend on any Emacs customization variables or hooks, and if it has to make any changes to Emacs’s state or state external to Emacs (such as the file system), it should undo these changes before it returns, regardless of whether it passed or failed.
Tests should not depend on the environment because any such
dependencies can make the test brittle or lead to failures that occur
only under certain circumstances and are hard to reproduce. Of
course, the code under test may have settings that affect its
behavior. In that case, it is best to make the test let-bind
all such setting variables to set up a specific configuration for the
duration of the test. The test can also set up a number of different
configurations and run the code under test with each.
Tests that have side effects on their environment should restore it to
its original state because any side effects that persist after the
test can disrupt the workflow of the programmer running the tests. If
the code under test has side effects on Emacs’s current state, such as
on the current buffer or window configuration, the test should create
a temporary buffer for the code to manipulate (using
with-temp-buffer), or save and restore the window configuration
(using save-window-excursion), respectively. For aspects of
the state that can not be preserved with such macros, cleanup should
be performed with unwind-protect, to ensure that the cleanup
occurs even if the test fails.
An exception to this are messages that the code under test prints with
message and similar logging; tests should not bother restoring
the *Message* buffer to its original state.
The above guidelines imply that tests should avoid calling highly
customizable commands such as find-file, except, of course, if
such commands are what they want to test. The exact behavior of
find-file depends on many settings such as
find-file-wildcards, enable-local-variables, and
auto-mode-alist. It is difficult to write a meaningful test if
its behavior can be affected by so many external factors. Also,
find-file has side effects that are hard to predict and thus
hard to undo: It may create a new buffer or reuse an existing
buffer if one is already visiting the requested file; and it runs
find-file-hook, which can have arbitrary side effects.
Instead, it is better to use lower-level mechanisms with simple and
predictable semantics like with-temp-buffer, insert or
insert-file-contents-literally, and to activate any desired mode
by calling the corresponding function directly, after binding the
hook variables to nil. This avoids the above problems.