3.7 The SUnit testing package

SUnit is a framework to write and perform test cases in Smalltalk, originarily written by the father of Extreme Programming¹⁶, Kent Beck. SUnit allows one to write the tests and check results in Smalltalk; while this approach has the disadvantage that testers need to be able to write simple Smalltalk programs, the resulting tests are very stable.

What follows is a description of the philosophy of SUnit and a description of its usage, excerpted from Kent Beck’s paper in which he describes SUnit.

3.7.1 Where should you start?

Testing is one of those impossible tasks. You’d like to be absolutely complete, so you can be sure the software will work. On the other hand, the number of possible states of your program is so large that you can’t possibly test all combinations.

If you start with a vague idea of what you’ll be testing, you’ll never get started. Far better to start with a single configuration whose behavior is predictable. As you get more experience with your software, you will be able to add to the list of configurations.

Such a configuration is called a fixture. Two example fixtures for testing Floats can be 1.0 and 2.0; two fixtures for testing Arrays can be #() and #(1 2 3).

By choosing a fixture you are saying what you will and won’t test for. A complete set of tests for a community of objects will have many fixtures, each of which will be tested many ways.

To design a test fixture you have to

• Subclass TestCase
• Add an instance variable for each known object in the fixture
• Override setUp to initialize the variables

3.7.2 How do you represent a single unit of testing?

You can predict the results of sending a message to a fixture. You need to represent such a predictable situation somehow. The simplest way to represent this is interactively. You open an Inspector on your fixture and you start sending it messages. There are two drawbacks to this method. First, you keep sending messages to the same fixture. If a test happens to mess that object up, all subsequent tests will fail, even though the code may be correct.

More importantly, though, you can’t easily communicate interactive tests to others. If you give someone else your objects, the only way they have of testing them is to have you come and inspect them.

By representing each predictable situation as an object, each with its own fixture, no two tests will ever interfere. Also, you can easily give tests to others to run. Represent a predictable reaction of a fixture as a method. Add a method to TestCase subclass, and stimulate the fixture in the method.

3.7.3 How do you test for expected results?

If you’re testing interactively, you check for expected results directly, by printing and inspecting your objects. Since tests are in their own objects, you need a way to programmatically look for problems. One way to accomplish this is to use the standard error handling mechanism (#error:) with testing logic to signal errors:

2 + 3 = 5 ifFalse: [self error: 'Wrong answer']

When you’re testing, you’d like to distinguish between errors you are checking for, like getting six as the sum of two and three, and errors you didn’t anticipate, like subscripts being out of bounds or messages not being understood.

There’s not a lot you can do about unanticipated errors (if you did something about them, they wouldn’t be unanticipated any more, would they?) When a catastrophic error occurs, the framework stops running the test case, records the error, and runs the next test case. Since each test case has its own fixture, the error in the previous case will not affect the next.

The testing framework makes checking for expected values simple by providing a method, #should:, that takes a Block as an argument. If the Block evaluates to true, everything is fine. Otherwise, the test case stops running, the failure is recorded, and the next test case runs.

So, you have to turn checks into a Block evaluating to a Boolean, and send the Block as the parameter to #should:.

In the example, after stimulating the fixture by adding an object to an empty Set, we want to check and make sure it’s in there:

SetTestCase>>#testAdd
    empty add: 5.
    self should: [empty includes: 5]

There is a variant on TestCase>>#should:. TestCase>>#shouldnt: causes the test case to fail if the Block argument evaluates to true. It is there so you don’t have to use (...) not.

Once you have a test case this far, you can run it. Create an instance of your TestCase subclass, giving it the selector of the testing method. Send run to the resulting object:

(SetTestCase selector: #testAdd) run

If it runs to completion, the test worked. If you get a walkback, something went wrong.

3.7.4 How do you collect and run many different test cases?

As soon as you have two test cases running, you’ll want to run them both one after the other without having to execute two do it’s. You could just string together a bunch of expressions to create and run test cases. However, when you then wanted to run “this bunch of cases and that bunch of cases” you’d be stuck.

The testing framework provides an object to represent a bunch of tests, TestSuite. A TestSuite runs a collection of test cases and reports their results all at once. Taking advantage of polymorphism, TestSuites can also contain other TestSuites, so you can put Joe’s tests and Tammy’s tests together by creating a higher level suite. Combine test cases into a test suite.

(TestSuite named: 'Money')
    add: (MoneyTestCase selector: #testAdd);
    add: (MoneyTestCase selector: #testSubtract);
    run

The result of sending #run to a TestSuite is a TestResult object. It records all the test cases that caused failures or errors, and the time at which the suite was run.

All of these objects are suitable for being stored in the image and retrieved. You can easily store a suite, then bring it in and run it, comparing results with previous runs.

3.7.5 Running testsuites from the command line

GNU Smalltalk includes a Smalltalk script to simplify running SUnit test suites. It is called gst-sunit. The command-line to gst-sunit specifies the packages, files and classes to test:

-I

--image-file

Run tests inside the given image.

-q

--quiet

Hide the program’s output. The results are still communicated with the program’s exit code.

-v

--verbose

Be more verbose, in particular this will cause gst-sunit to write which test is currently being executed.

-f FILE

--file=FILE

Load FILE before running the required test cases.

-p PACKAGE

--package=PACKAGE

Load PACKAGE and its dependencies, and add PACKAGE’s tests to the set of test cases to run.

CLASS

CLASS*

Add CLASS to the set of test cases to run. An asterisk after the class name adds all the classes in CLASS’s hierarchy. In particular, each selector whose name starts with test constitutes a separate test case.

VAR=VALUE

Associate variable VAR with a value. Variables allow customization of the testing environment. For example, the username with which to access a database can be specified with variables. From within a test, variables are accessible with code like this:

    TestSuitesScripter variableAt: 'mysqluser' ifAbsent: [ 'root' ]

Note that a #variableAt: variant does not exist, because the testsuite should pick default values in case the variables are not specified by the user.