Unit Testing

In a well-designed product, every object you create will have been designed and modeled. The requirements of this object should be captured in a collaboration diagram, and so it should be possible to understand and to test all of its functionality. When you code the object, you will also want to code a test-harness. A test-harness is a body of code designed to test the functionality of the object, but which will not ship in the final product.

A test-harness lets you test the client in the absence of the server; it lets you test each object individually and allows you to control and log the internal state of the object. This ensures that the object only moves through legal internal states, and is never left in an indeterminate condition.

Typically the test harness will be provided as a hidden menu choice, which the developer can use to invoke specific methods on the object and get back validating information. A well-designed test-harness will include a suite of tests, which can be run individually or as a set, one time or repeatedly. A first class test suite will include the ability to kick off a set of tests to be run repeatedly and will provide a comprehensive log of its results. This facilitates regression testing. It is imperative to test the entire product after each change, however unrelated that change appears to be. Automated tests facilitate regression testing, as they overcome the temptation to cut corners.

The various states of each object will be modeled in state transition diagrams, and each of these transitions should also be testable. Your test harness will move the object into and out of various states, confirming that the integrity of the object is maintained through the transitions, and, equally important, that only legal transitions are possible.

Every interaction between this object and all other objects in the system should be modeled in the collaboration diagrams. These interactions will be tested in higher-level interaction tests; again, as part of a test-harness. You may build test suites which test each object individually, and then test various interactions among the objects, documenting the results of these interactions.

Building the test-harness is a big job, and one that should be designed in advance, in cooperation with whomever will be doing the quality assurance (QA) of the product. The testing software will be used by QA throughout the development cycle — you should avoid the temptation to pass your product to QA as a final step in development. The waterfall method doesn't work in quality assurance any better than it does in development; what you want is a series of small iterations, in which features are designed, developed and tested.

An additional benefit to building these tests as you go along is that they require you fully to understand the properties, methods and interactions of each object. Developing the test-harness forces you to think through all the methods and their potential results, and this can feed back into your design and model, creating a more complete model of each object and its interactions.

Testing is a big subject; doing it justice could fill a book of equal size. In addition to unit testing you must put your system through extensive integration testing (proving that the components work together). You must then turn to performance testing, to ensure that you've met the performance requirements you documented in your architecture document, and finally you must put the system under load— testing its ability to perform under stress and to scale to realistic conditions.

Finally, when all of this testing is complete, it is time to put the code in front of "friendly users" — alpha testers who will give you preliminary feedback on real-world use of the application. Assuming that you survive alpha-testing, it is time to go to beta-testing; that is testing with real-world users who don't know you and have no reason to be particularly kind in their review of the product. The significant difference between alpha and beta testing is that alpha testers know you and understand that they are using a fragile and nascent product. Beta testers know they are giving the final imprimatur to the product, and are apt to be harsher in their review.

If you get through beta testing without finding "show stopping" bugs; it is time to go to FCS: First Customer Ship. Shrink wrap it, send it out to tens of thousands of users and wait for the reviews. If you are lucky, the checks start rolling in. If you were sloppy, the bug reports roll in. And if you are dreadfully unlucky you will be met with a deafening silence.

© 1998 by Wrox Press. All rights reserved.