Monday, August 24, 2009

Platitudes Of Doom

Here's where I suspect things go wrong. I see a combination of two ideas at the core of a lot of corporate dysfunction.

  1. I can want things faster than you can build them.

  2. You can always do more if you really try.

The former is obviously true. Wanting is cheap, instantaneous, and unencumbered. I want a car with a 500K mile all-inclusive warranty, Maserati looks and performance, moped operating costs, and a Hot Wheels sticker price. And I want it to drive itself. I am betting that the average reader had at least three improvements they would make to my wish list. None of us expect to see such a thing, of course. The engineering and design costs and material costs make this impossible.

Wanting is cheap, but invention and production are expensive. There are material costs, costs of design, costs of errors and rework, costs of research, time and materials. There are human beings who do design, and they like to be paid. It takes them longer to work out the requirements than it does for me to decide I would like a new kind of car, or desk lamp, or smart phone. Again, it is the very nature of work that invention is generally much more expensive than desire.

Now, an astute reader will see the first statement as an axiom. I should adjust my desires and decide which things I really, really want first. I should realize that all the things I want will take time, and perhaps more time than one would expect. I should not be surprised that I will want many more things before the first thing I desired is complete and delivered. I should realize that many of the things I want are beyond the capacity of the market to provide (at least for now). I should realize that I may have to sacrifice some less important desires (however achievable) to achieve some greater ones. I should expect to be disappointed if I do not adjust my expectations.

In software development, the primary input to today's efficiency is the quality of the code we have written until now. If that code is clean, bug-free, well-designed, and well-organized then today's task will be reasonably estimable. If yesterday's code is full of confusion and scant on tests, full of ugliness and poor organization, then today's work cannot be reasonably well estimated (even with 250% margin of error). Yesterday's low quality is today's encumberance, or maybe I should say that todays technical debt means we have less technical capacity to spend every day until we pay it off. Note: there was a nice talk on this topic by Jim Highsmith, mingled into a talk on quality metrics. And I suppose it is a topic for another day.

On the other hand, less astute reader will take this as a problem statement, and focus on the fact that the implementation team is, for some reason not delivering the things we want fast enough. This is where the second statement comes in. If people can always get more done by trying harder, then my desires are unmet because they don't care enough since they aren't really trying.

The sad part is that the second statement is false.

Inventors and designers have frequently worked themselves past a point of diminishing returns. A tired, cranky, unfulfilled, pressurized worker does not become better by trying harder. Trying harder only works a little bit, and only for a little while, and only if the developer was not really trying hard to begin with. After the frustration sets in, trying harder just makes it worse. Sometimes you need a more orderly approach, and not a more passionate one. Sometimes you need some new skills and techniques, not a greater display of sacrificial devotion. Frankly, it doesn't even make sense that the worker would sacrifice all in devotion to a harsh taskmaster.

That second statement rephrases technical issues as motivational issues. It takes a problem from a context in which it can be managed (quality, technique, capacity, and priority) and into a context in which it cannot be solved but can be easily worsened (Taylorist motivation). It amplifies a misunderstanding of the first statement in a particularly bad and quite synergistic way.

If a team can not achieve more by working harder, or will not produce as well, then pressure is a poor choice.

My colleague in practice, Darrin, brought up Taylorism and Anti-Taylorism over at Agile In A Flash. Of course, a large number of talks at Agile 2009 are on managing backlogs, improving quality, and dealing with productivity. The meme is in the air (as it has been since before Mythical Man Month). I wonder how many development problems we can solve if we can just accept the first statement as an axiom and recognize the second as a falsehood? How can we bring this about?

I suspect we will need to develop some kind of "desire management" discipline. What would its tools be?
  • Value assessment. We should get past "want" and "contract" and into issues of value. What features will most improve our standing in the market? What features will most improve our reputation? Which features will cheapen us? Which yield little or no ongoing improvement? Which features will never really be used (AKA "checklist features")? Which should we farm out? Which should we partner out? Which should we forget about? and which should we really be spending our time implementing?

  • Value Alignment. In particular, we need to be more interested in what our customers value. The backlog has to stop being a way that our sales/product/management staff compete against each other and become a way that we improve the value of our company and our product in the market. We need to want better things.

  • Cost assessment. We need to quit trying to drive estimates downward, and take them under advisement. Even today, most estimates are ambitious and not conservative (more likely to go long than fall short) and yet we try to push them down to a cost we want rather than adjusting our desires to be within our budgets. That needs a rethink.

  • Expectation management. We need to get the truth about capacity, WIP to the decision-makers in our organizations. None of our companies have unlimited capacity, and lean concepts are the only way we know to really get the best out of our people. The system has limits, and we need to respect them first and adjust them second. We cannot simply ignore them.

  • Controlled disappointment. We need to stop stringing our customers along. Sales, marketing, and developers alike have a nasty habit of wishful thinking. They put things on the backlog that are not realistically going to be completed in the next month, the next year, or the next three years. I might suggest honesty instead of delay tactics. This work is not going to be done any time soon. Will your customers be happier and respect you more if you keep telling them it will? I think they'll be disappointed to find the work won't be done, but moreso if they find out after a series of false assurances to the contrary. Controlled disappointment might be a good policy.

  • Operational cost assessment. Maybe we should not consider a $2M contract a win if it will take $1M to build the feature, and will increase operating costs by $10K per month for the indefinite future, and implementing the feature will cause us to lose $3M in opportunities as other features are delayed. ROI is not all about the money we can make on a contract, and development efforts go into the red more often than anyone wants to admit.

The secret, we're told, is to:
  • do less
  • do it better, and
  • do it more often.
That sounds easy, but it is swimming upstream in many organizations. How can we make it better?