This week, and last, we are doing our work in the release candidate (RC) branch, which will eventually be merged to trunk. We maintain a "stable trunk" system, with the RC as our codeline (for now). This is an intermediate step on our way to continuous integration.
Partly because of the change in version control, the team has learned to rely more upon the tests, and is writing them quickly. We have had a noticeable increase in both unit tests (UTs) and automated user acceptance tests (UATs) in only one week There were some problems with people checking in code for which some tests did not pass, but they have learned very quickly that this is quite unwelcome.
We are painfully aware of the time it takes to run both test suites. The UTs suffer from a common testability problem, in that they were written to use the database and they sometimes tend to be subsystem tests rather than truly unit tests. When they are scoped down and mocking is applied, they should be much faster. Sadly, we are using one of those ORM frameworks that wants to own our objects and bind them tightly to the database, so we will have to go through some more mechanizations to get our objects free of the database. This is common, but always troublesome. The features that make a framework convenient can be the same ones that frustrate all attempts at building moderately comprehensive test suites. Our unit tests take over 10 minutes on my computer, and the UATs take much longer. =8-o
We have been closing down old branches for a while now (releasing backlogged work), which can only increase our productivity by decreasing the "drag" of branch maintenance and troublesome integrations. We have not outlawed development branches, but we will start committing to a small amount of work to always be done in the RC, with larger tasks or those with uncertain results branched for now.
We have a nosetest-based harness for gathering coverage information from unit tests, and I hooked up coverage.py to collect the same data for our UATs. It's not a perfect system yet, but we can at least start to chart some trends.
Our Continuous Integration effort is nascent. I'm going to try to find a way to set up buildbot to run all our unit tests (at least) and then to launch the UATs through FitNesse (always a pain to automate). I'm expecting a lot of fun here.
Our informative workspace initiative is coming along. We have UT counts and timing graphs, the same for UATs, working card-walls for our tasks, simple process information, etc. Some of our programmers have been producing monthly production charts to track the amount of money moving through the system, etc.
Overall, we're doing a pretty good job of transitioning. We have challenges, but we've come a long way.
No comments:
Post a Comment