Thursday, June 24, 2010

Two Year In, Looking for the Goodies

This week I wrote a useful utility and taught some Python to a peer, attended a really great work session and a rough estimating session. I did some WIP management activities with my team, I broke the application, I worked with some awesome partners, went on a bit of a refactoring binge, devised (with partner) a grand strategy, and got stuck on tiny little problems I couldn't see over. This has been an awesome week.

We have been doing some big refactorings, moving big and critical chunks from one module to another, and dealing with the fallout. I don't know how to tell you how important the tests in our code base have been. I would have spent all day lost without them. We would move a component, build, resolve build troubles, rinse and repeat. Finally, at end of day, I got all unit tests and fitnesse tests turning green, EXCEPT for a few that won't test because they do horrible things and need remediation. When our tests failed, it was not because the tests were bad (exception, one was pretty iffy) but because something was wrong. The compiler and tests made our work less scary and troubling. It was still scary and troubling, but would have been overwhelming otherwise.

I and some of my colleagues transitioned this team to Agile only 2 years ago. It is amazing how deeply some of the team members grok Agile now, and a little frustrating how some people and some circumstances have not really come up to the same level. Frustration is not failure, though. There is light bursting through here and there.

I've also been checking in as a coach, testing the prevailing winds, looking for the problems that keep us down, looking for the ways to improve our work system, looking for moods issues. We're in a pretty good place, actually, by numeric measures. Some parts of the code base are very well-tested and quite simple, and others are becoming so. There is still some amount of legacy code produced every day, though, and that is a bit disheartening to those who really want to see only improvement (I'll cop to being one of them).

Our biggest challenge is that there is some big work coming to some ticklish areas of the code, and we'll be under pressure to deliver. This is our first real chance to test our Agile mettle against a really formidable foe.

Jeff and I just released the "Why Agile" card at AgileInAFlash. Two years into the transition, I want to make some comparisons on how we rate today v the original situation.

Improve Customer Involvement has improved the least. We have a great guy playing the role of Customer, but largely the representatives for the customer don't join the team. Some of them have found more engaging ways to provide specs (screenshots, videos, etc), and have made themselves more available for discussion. It is still not what it could be. With one notable exception, our Customer players are all surrogates. All but one of the surrogates live on the other side of the building, across a hallway from the developers, and are not really involved in daily work. This is actually a problem that affects morale and development performance. Finding a way to bring us all together would bring profound changes to the product, I have no doubt.

Increase Quality has come a long, long way. Like any SAAS player we always want better performance, and like any SMB growing larger we have a few issues from time to time, but overall quality is way up and the testing practices are better all the time. When I ask product management or sales or support, they tell me that it is a whole different world from the product of 2008. We haven't reached the "everybody loves us" stage yet, but we have come a long way.

Simplify Releases is happening. They used to have nightmares about release nights, and now they're more like annoyances. We still have code freeze and pre-release ceremonies, but we're not 100% covered in tests yet, and we put a lot of features in each month. We will continue to need some regression testing and some exploratory testing for a while yet. Issues that made code hard to deploy keep getting eliminated. It's a new world compared to our starting position.

Increase Operational Awareness this one hasn't come so far. We don't have the information radiators I would like to see, but the card wall is being used, and people tend to have a pretty good sense of what is in progress, finished, or unstarted. The big priorities are posted on pillars and doors so we can see them. Stand-up meetings help us keep track of each other. It could be better, but it is a good start. Actually, the operations staff have herereally smoked us in this regard. We know more about servers and their problems than ever before. We have moving graphs and charts updating constantly on large-screen displays in the office. There are alerts in email, and people stopping by the developer pairing stations. They're doing a tremendous job.

Drive Down Risk could be better. We tend to fall back to producing whole big features, rather than staying incremental as we should. I know, breaking down stories into valuable verticals is tough business, but all-or-nothing leaves a strong chance of "nothing." Our demos do collect feedback, and often are valuable working sessions (ours was this week). Testing and pairing also help reduce developer myopia and its resulting code diseases. But still, we're not incremental as we should be, and that leaves us at risk more often than I'd like.

Reconnect With Geek Joy is there, but we still wrestle with this. Sometimes we don't feel better, even though we are better. A times wrestling with legacy code (even new legacy code) frustrates us. Sometimes we don't have the level of success we might like. Yet I am so encouraged when we can do big things under sufficient tests, and when we find people making far-ranging improvements, when we have consensus on big design changes, and when we demonstrate working code. There is energy around performance and quality. We enjoy solving problems, and we enjoy cleaning code, and we take pride in not leaving silly duplication behind. I enjoy this job, and am happy to be doing it. We are making a difference to each other and serving customers as better than we could before. This job improves me, and that's joyful.

I know, my readers may want to hear how I totally transformed the company and how all of our problems vanished in 6 weeks or less, but that's not always how it goes. Real change takes time, and not everyone "gets it" all at the same time. New hires, old cliques, old barriers, old policies, old politics, new features, new customers, and new technology keep things challenging. Pressure to release pushes people back into old bad habits. There is a swirling cloud of chaos, but it keeps moving in a desirable direction overall.

Maybe we're still struggling a bit, but that's what growth is like. Watch this space. Maybe next year I'll show you how much further we've come.

Tuesday, June 22, 2010

Do You Want Us To Think Or Not?

To make software, you need people who are skilled, creative, quick-learning, thoughtful, and engaged. You might be able to skimp on 'skilled' if the rest of your team provides a social learning system (via pair programming, test-driven development, book and link sharing, impromptu teaching, design talks, etc). You cannot really skimp on the characteristics of being creative, thoughtful, and engaged.

If you tell someone that you don't care how they feel, they will work in an unfeeling way. If you tell them that you don't care what they think, they will work unthinkingly. If you tell them not to participate, they will disengage. This (IMHO) is why use of corporate software is so often without soul and without joy. If one self-important side tells the other to shut up, or to stop asking questions, or to leave the decision-making process, then he has switched off an important piece of capital equipment.

If you treat a knowledge worker as a drudge, she will turn off her brain and trudge. Unsurprisingly, she will lose energy, become uncaring toward the product, and start phoning in her minimal-thought hacks while polishing off her resume. It is hard to keep good people in a system that doesn't really value them.

In an Agile company, we expect customers (or a reliable surrogate) to team up with developers. We expect to have a two-way flow of communication. Each side asks questions, provides answers, researches, considers. Each side teaches and learns. In time the whole team (customers+developers) will take on creative problem-solving, will improve the user experience, will make the product data-center-friendly and user-friendly, and will produce truly valuable work.

All of the software development roles are creative. All of them have issues and difficulties that the other roles may not understand initially. But if we're going to do good work, we have to learn to listen to each other. All it takes to destroy the magic is for one to devalue the contributions of the other.

A few resources:
From Agile Software Development, an article on respect.
Esther Derby's Stop Demotivating Me from CIO magazine.
Daniel Pink's talk via my blog.

Friday, June 18, 2010

Meddling, Oversight, and Agile, Oh My!

A friend of mine (Hi George D) suggested that this would make a good poster,
but all I have is a couple of blogs, so here is the message that inspired
my buddy.

My experience is that the less well a team has done in the past,
the more oversight is piled on, and that oversight reaches higher
and higher levels. There really is no legitimate reason for the
CEO to want to know which programmer 5 or more levels below was
assigned to a particular task and if he's behind schedule by a
week or so.

In healthier organizations, the groups and managers that interface
with the development group tend not to have the same meddlesome urges.

In our transitions, the biggest problem we face tends to be peeling
back the expensive and unnecessary oversight. If the team can be
rebooted and work with a single stream of smaller, simpler stories
(rest of agile practices included) then they can win over the
rest of the org in relatively short order. Sometimes in only a
year or two, sometimes in less than 6 months.

It's hard to overcome meddlesome urges, especially when they are in
reaction to difficult past history (and moreso if there is competition
among the meddlers to get their work done).

Tools play into that. People want tools partly so that they can
pile on too much work (more than fits in a head or on a board)
and partly so that they can meddle from the comfort of a remote

Agile is, among other things, a very intense and orderly way to
get work done. It's just hard to get the focus of a conflicted
organization on getting work done in an intense and orderly way.

Only Quality Matters

If your code doesn't really work, then it doesn't matter how quickly you wrote it.

Abandoning quality in pursuit of speed is a fool's errand. It has been discussed plenty of times in about any management book you want to look at, and especially by the brilliant minds like Deming.

If you write your code badly, you can release it without awareness of bugs, and that takes it off of your plate. You may be rewarded or recognized for your quick work. It feels good today. Your team is recognized for being heroes within the company because you are smart, and get things done.

But tomorrow a customer tries to use it and it fails. The customer is no longer happy that the feature was delivered, because it's junk. In fact, as far as the customer is concerned the whole product is now junk, your company is junk, and the developers are pure rubbish. Loss of good will. Think how you feel when you get the Fail Whale, and you are a friendly customer and unusually forgiving. How do you feel when your cable provider or cell phone service has a hiccup, or when your windows mobile phone has to be rebooted. Normal people are much less forgiving.

Now that failure is reported (if you're lucky) and someone in product support hears about it. They feel less proud of their affiliation. They push to get a change made, but the programmers 1) are busy on something else, and 2) take a long time to find the bug. The product support queue grows. People are employed now to track mistakes and to try to run interference with unhappy customers.

The failure isn't just reported to the vendor, but to all the customer's bosses, who begin to question their vendor relationships. A few bugs now and then, they might be able to tolerate. but constant performance and behavioral bugs will have them holding the contract in one hand and a lighter in the other.

People are employed to fix the bugs, and to test the code to be sure the bug is gone. A few tests written TDD-style would have saved a lot of work, money, and hard feelings. But you were in a hurry.

If the code doesn't really work, then it does not matter how quickly you wrote it.

Tuesday, June 15, 2010

Software Like Building A House

After arguing for decades that it is not so, I find out that I'm wrong. This is from a 2007 article on construction.
1. If it takes six months to build a house, then 85 percent of the time is spent on two activities: waiting on the next trade to show up and fixing mistakes.

2. Clemson's Professor Roger Liska conducted an analysis of productivity on the construction industry and found that the average construction worker operates at only 40 percent efficiency.

3. Critical shortages exist in qualified, skilled workers and labor issue futurist Roger Herman predicts the situation is only going to get worse.

4. Business Week's 2007 Investment Outlook Report indicates the return on equity (ROE) for all U.S. industries is 17.9 percent, while the ROE for the construction industry is a mere 9.7 percent, despite the recent construction boom.

5. Industry customers are frustrated with poor quality, confrontation, excessive change orders in quantity and dollar value, scheduling delays and litigation.

Thursday, June 10, 2010

Your Reward System Doesn't Work

Presented in the most pleasant way possible, by Daniel Pink & friends:

The whole "purpose maximizers" thing gives me hope in ways that "enlightened self interest" never, ever has.