EGCS 1.1 regression test strategy

One of the critical steps in making releases is regression testing. For EGCS 1.1 we want to perform regression tests against GCC 2.8.1 and EGCS 1.0.3a.

Since it is impossible to test every platform, we have selected a list of popular platforms for which EGCS 1.1 should have no regressions relative to EGCS 1.0.3a or GCC 2.8.1.

We encourage testing of platforms not on the official regression test list. However, the "no regressions" policy does not necessarily apply to targets not on the official regression test list.

Contact law@cygnus.com directly if you want to help with the regression testing phase of the EGCS 1.1 release cycle.

How to run the regression tests

First, you must get out of the EGCS object directory. So change directories into your home directory, /tmp or somewhere else.

Second, you need an appropriate site.exp. Here is a template:

# snapdir should point to the top-level directory of an unpacked EGCS 1.1
# snapshot
set snapdir "/some/place/EGCS 19980715"

# set old_release_prefix to the prefix directory of an installed EGCS 1.0.3a.
set old_release_prefix /prefix/for/EGCS 1.0.3a

set rootme "."
set target_triplet [exec "$snapdir/config.guess"]
set srcdir "$snapdir/gcc/testsuite"
set GCC_UNDER_TEST "$old_release_prefix/bin/gcc"
set GXX_UNDER_TEST "$old_release_prefix/bin/g++"
set G77_UNDER_TEST "$old_release_prefix/bin/g77"

Obviously you have to change "snapdir" and "old_release_prefix" to appropriate values.

Then run the tests with the following commands:

  runtest --tool gcc
  runtest --tool g++
  runtest --tool g77 (Only necessary if you have installed g77)

This will create "gcc.sum", "g++.sum" and "g77.sum". You should send those files to the testing coordinator law@cygnus.com.

Note that you will receive many failures from GCC 2.8.1 and EGCS 1.0.3a when running the gcc testsuites because those versions of the compiler do not support -Os.

Current Regression Testing Status

*Note Everything is failing gcc.dg/980626-1.c. It's a bogus warning test, so I don't consider it critical, but I have asked for a volunteer to look into the problem.