Tuesday, May 14, 2013

Invisibility of Process, Visibility of Results

There are some special challenges with dealing with productivity of knowledge workers. Most of them have to do with the invisibility of the work and the difficulty in managing invisible work.

Programmers and testers don't assemble machinery or bend paperclips or mold parts from molten goo. They don't stack boxes or bricks, or swing hammers. The work they do has no physical manifestation, which makes it both hard to observe and hard to understand.

I don't blame managers in the 70s and 80s who counted lines of code. It was one of the few visible manifestations of the work programmers do. It was entirely misguided of course, and several of us have experienced net-negative lines of code in consecutive weeks of work (I've even had awkward and unpleasant meetings with managers for "messing up the metrics," ending in an admonition to stop it).

Other attempts to make the work visible count data fields on screens and in databases and on reports. This is a bit better, because it deals with the visible aspects of the system, not the bulk of the implementation. My one experience with function points was on a multi-year embedded control system which, based purely on data elements and screens, should have taken a few weeks to produce -- the function point measuring system at that time did not count decisions, constraints, or the number of operating-system workarounds involved in getting performance out of the system.


Let's not overlook the other knowledge workers we see on a daily basis.  Management is also largely invisible and depends so heavily on intangibles that it's often hard to separate out the qualities that make managers effective compared to the ones that drive teams to create messy piles of junk.

Our managers much work in ideas, processes, directives, initiatives, relationships, impressions, signaling and messaging, systems, and concepts. 

Management has a lot of the same problem as programmers. It's hard to see the full range of effects that decisions might have.

Even a sigh, a snort, or an ill-timed change of body posture can have unfortunate unanticipated consequences for a manager. Consider the Moz story "He's not going to get very far" as an example of how developers interpret small clues as major directives.

Programmers and managers are in the same kind of pickle.

Our work is intellectual and invisible on both sides of the gap, and we rely on a relationship informative feedback cycles to improve our performance.

If "stuff that works" is our goal, then we need to return to the ideas that make it possible to build stuff that really works and really solves problems for people in the real world.


So if software is not best measured in lines of code or data elements, how can we understand the software process better? How can we create a better theory of productivity for a largely invisible system?

The gold standard of XP teams is "running, tested features (RTF)."  It considers the value-producing part of the system (features), rather than the cost-inducing part (code). Even though features are not of uniform size and scale, we know that more value-producing parts is better than fewer.

Sadly, a lot of shops are still working on a system of pre-made promises that stretch well into the future. If we are producing a lot of running tested features but we are not on track to make all the promises that have been made by other people, then we start to deal with the pressure to produce in a less healthy way.

The ratio of things that the company says it will do compared to the things it can actually do is a crucial point of reputation. Are we dealing with under-delivery or over-promise? What is the best result we can expect when the say:do ratio of the company is out of proportion?

The wrong answer is to crank up more metrics to show how busy we are. Sadly, the common "answer" in many companies is to do exactly this, and to fall back to the naive theory of productivity. Why? Because it's the best we know how to do in the circumstances.