Linear and Radial Progress: Take The Next Step

Today we didn't make a lot of linear progress

I think of quality in two ways.  One is the product quality that users perceive, and the other is the habitability of the code itself. One says whether the function points and UX are well-selected, and the other says whether it is worth your time to try to work in the code.  One can have crap code that seems great externally, and one can have beautiful code that doesn't do anything a user is interested in doing (the popularly presented dichotomy). Likewise one could have poor functionality written poorly, or great functionality written well.  Those who consider themselves professionals in software craft will desire great functionally written well.

I often get to work in code that started out as good functionality badly written. I get to work on the teams that clean up the mess and make the system more useful, scalable, and performant. As we work, the code gathers external quality. It becomes increasingly virtuous.  Smaller, simpler, cleaner code starts to run faster. With less duplication, bugs are fixed once-for-all rather than as buggy duplicates are discovered. This month is better than last month, but next month should be better still.

Because there are two kinds of quality, and one can make progress on either front, we see two kinds of progress. There is linear progress toward completion of a specific feature, and then there is radial progress which improves the code base overall and makes work easier for many colleagues.

Developers who make linear progress without radial progress ultimately drive code into an increasingly unliveable state. Eventually, bugs are hard to resolve, changes are hard to make, testing is costly, refactoring is painful, and the design is more a collection of warts and hacks than a system any sane beings would have intentionally conceived.
 
Radial progress does not contribute much to one's ability to close a particular development request. Instead, it improves the ability of the team to complete features next month and the months after that. It tends to affect many source files, so it is uneconomical in projects that don't have a significant investment in automated testing.

We made a lot of radial progress today.

We started with a change we wanted to make, but instead of slapdash hacking we took The Next Step and looked at the tests that covered the area of change. We found the unit tests were poorly written. These tests had poor fault isolation and failed for the wrong reasons. This prompted us to take The Next Step and make them less fragile so that they would stay 'green'.

From reverse-engineering the unit tests, we saw that the code under test was nigh indecipherable. It was riddled with poor naming, inconsistency, huge argument lists, lying comments, and slapdash implementation.  Support routines did little to clarify the code, and in come cases further obscured its intent. Tests were fragile and indirect.

We decided to take The Next Step by refactoring some tests for readability. We replaced some loops with LINQ, after which it was obvious that the code had unnecessary looping. Each readability improvement showed some error in linear thinking that almost certainly slowed the initial development of this part of the codebase.

Writing bad tests is hard work. It would have been easier to write them better or to refactor them as they went, but they were almost certainly more focused on making linear progress than on making the code clean. I can't really blame them. People under pressure cut corners and sometimes "it works" is all they think they can afford. People do the things that make them successful, and sometimes they are rewarded or punished for the wrong things. As is, it suffices to say that they did hard work the hard way, and so we found it in a poor state of development.

The more we introduced explanatory variables, simplified expressions, and eliminated duplication, the more we could see the next set of flaws to address.  We took The Next Step and started reworking the test harness. Soon we find that there are variables being created and passed from routine to routine only to be ignored completely at the lower levels.  We cleaned up the Object Mother utility, obviating hundreds (if not thousands) of unnecessary lines of code, and I suppose we should take The Next Step and remove the unnecessary code from the ObjectMother users.


This sounds like weeks of work but was only hours. We closed one ticket, opened another, attended a long planning meeting and a lunch-and-learn. We took breaks for all the usual reasons. We broke up our pairing session at 5:00 CDT.  We understand the code much better, and the tests explain the code better.

Sure, the code cleaning could turn into a few man-days of effort, but it is an effort that will benefit everyone who wanders into this area of the code in the future.  This area of the code base is being actively developed and has outstanding bugs, so we will likely recoup all our investment in the course of completing very few change requests.  We are aware that we can't afford to spend time gold-plating this module, but we can make it habitable before we move on.

On the XP mailing list, someone asked what distinguishes a truly great agile development team. Let me offer that the difference is that a great team will take The Next Step to make the code habitable on the inside in addition to making it work.  Because of this tendency, each month gets better, each quarter more productive. Each year, the system requires less patience of all those associated with it, and provides more rewards.

That, my friends, is great.

Late addition: If course Ron Jeffries wrote about this first, and better.