How to Minimize Test Cases for Bugs

In order for a GCC developer to fix a bug, the bug must be reproducible by means of a self-contained test case. Our bug reporting instructions ask for the preprocessed version of the file that triggers the bug. Often this file is very large; there are several reasons for making it as small as possible:

There are basically two methods: either directly construct a new testcase that reproduces the bug, or iteratively reduce the original testcase. It is possible to automatically reduce testcases. Most test cases can be reduced to fewer than 30 lines!

Direct approach

Always try a direct approach before resorting to the brute-force method of minimizing a large test case:

Brute force approach

The brute force approach is simply an iterative method to cut down on the size of the test case by deleting chunks of code. After each attempt to shorten the test case, check whether the bug still is evident. If not, back out to the previous version that demonstrated the bug. For this, either use the undo function of your editor, or use frequent backup files; alternatively, you can use #if 0 and #endif. The automatic and recommended way to reduce a testcase is using the C-Vise or the C-Reduce tool.

If you have access to the original sources, it is better to start with the original sources and when those cannot be reduced further, generate and reduce the preprocessed sources. Both methods are described below.

Stripping the original code

Copy the original source file, plus header files that you might need to modify, to a separate location where you can duplicate the failure. Try the following, backing up from the unsuccessful attempts.

Usually, the file will now be down to fewer than 30 lines, plus includes. Repeat the same steps with remaining included header files. To reduce the number of files you are working on, you can directly include the content of the header file into the source file you are working on.

In the next step, run the compiler with -save-temps to get the preprocessed source file.

Stripping preprocessed sources

The preprocessed file contains lines starting with # that tell the compiler the original file and line number of each line of code. Remove these file and line directives from the preprocessed file so that the compiler will report locations in the preprocessed file, using one of the following:

The preprocessed sources will now consist largely of header files. Follow the same steps as for stripping the original code, with these additional techniques.

At this stage, you will be able to delete chunks of hundreds or even thousands of lines at a time and can quickly reduce the preprocessed sources to something small.