Friday, January 7, 2011

About "Under Test"

I wrote earlier about code being under test, and how advantageous to have a test "under" several layers of testing. As I was looking through my old google docs, I found this drawing I'd made to explain layers of testing in a web app.

Terminology is catered to the audience of the time, and will probably confuse some. For instance "controller" here is really more like the "model" layer with its duty of acting as facade and moderator for underlying objects. Its purpose is primarily ensuring business rules and performing a single broad function that includes potentially many objects at the lower level.

The "API" objects here are the business objects. Again, tailored to the local terminology. Apologies. Maybe the Agile Otter should redraw the whole thing sometime soon. Anyway...

The idea is pretty simple, but sometimes people don't quite understand why the code-behind needs to be thin and why they shouldn't re-implement functionality in the fitnesse tests. If we use good cohesion and manage coupling well, then the unit testing becomes fairly simple, as do the unit tests on the controller.

We can test the code-behind through the web but even automated browser-tests are slow, often running for hours. It is handy to have browser tests, but nobody wants to sit and wait for them to finish. Most shops will push them off to the build farm so they don't suck up developer time. If code it only tested by-hand through the user interface, testing is pushed off to a dedicated testing team who will script and re-run the sames tests month after month for hours. It quickly becomes too expensive to keep scaling that team, and they start selectively running ui tests, in which case errors will slip out to production and customer care will have extra work, and bug fix reports will pull programmers out of new feature engineering. Often a company will have to split off a separate full-time team to do production bug fixes and data repair. It gets expensive quickly. Automating the tests means that we might be able to run all the tests every day, so that fewer defects can get out to the customers. It's cost-effective, but again very slow. The less the code-behind does, the better we can deal with it being potentially untested.

Our integration and acceptance tests (shown here with FitNesse) are going to run many minutes, maybe an hour or more. If the tests take only a few minutes then programmers will run them several times a day, probably before committing or pushing code to the shared repository, almost certainly at the completion of a programming task. Ideally these tests will bypass the code-behind entirely, but not re-implement any of its functionality. In order to make these tests effective, we need to empty the code-behind so that it is little more than plumbing between the controller and the UI. The business function controller is pretty testable, but the tests are like integration tests with many objects involved and possibly significant effects. It can still be tricky to test at this level unless most of the interesting bigs are pushed down into the business objects. The less it does, the easier the testing is.

Finally we're getting into the individual business objects. These tend to be pretty easy to test, as they mostly act only upon themselves. Good news for us. We can write extensive tests to drive behaviors into the objects. They become robust and predictable and understandable to the degree we stick to what we know about coupling and cohesion. Because they are underneath layers of testing (UI, FitNesse, controller, and individual unit tests) we can have more confidence in the changes we make here.

Maybe the picture isn't perfect, and the terminology is a little twisted, but the idea here is very simple. It works out that we can save ourselves time and money and even save face if we learn to keep the upper layers of the application thin, and save the meaty implementation for the areas where it is under layers of testing.

Of course, all blessings are mixed and even dark clouds can have a silver lining, so simple ideas can have costs and side-effects all their own. But this simple idea seems to work and seems to reduce problems for development teams and companies which employ them.