Of course, this is a cross-cutting concern that deals with a big "ility." In particular it deals with scalability but I don't want to provide a bunch of detail that will distract us from the point.
The code is legacy in both the MFeathers meaning "without unit tests" an in the sense of "handed down from one generation to another, unsuspecting one." The new generation has done some excellent work getting huge tracts of land cleared and fenced with TDD and AT and what-have-you. Really, quite the transformation. The original designers had a philosophy and working style that did not survive the transformation (we think for the better) so there are architectural/design decisions being unmade on a regular basis.
In particular, there is this giant jellyfish of a design decision that's gotten in the way. It has long, long, long tentacles that extend far into the depths of layers of code, across the type system in funny ways, and into the realm of architectural concerns. When it's fixed, it will make the system better in many ways, and will clear the way to a whole host of other improvements. In short, it may be the coolest subproject in the whole company.
The jellyfish represents the munging of two or three separate concerns in one mechanism. It is a facility that was so amazingly handy that developers used it whenever they could. Remember that one man's fuzzy boundaries are another man's flexible solution. Now the concerns have to be split and the mechanism changed.
... one man's fuzzy boundaries are another man's flexible solution ...
We've managed to dredge up one stinging tentacle after the other, but there are still several more. In the course of doing so, we've had to make a branch (a short-term fork, really) and we spend a pretty significant amount of time merging code from the trunk.
I was commenting to a pair partner (Hi, Nick) the other day that we should have worked out a way to get this thing out in iteration-sized buckets. As soon as I said it I realized that we would have, had we known that the finished result was going to look like it does now/so-far.
This is not the first jellyfish I've met while swimming in legacy waters. In another company, Ed worked on a problem for an entire year and yet there were unexpected avenues in data access that still complicated the process. Not because Ed wasn't thorough and smart, but things can get out of hand politically and technically. Politics complicated the technical work, and there was little fun to go around.
I'm trying to recover and determine how we could have made these changes in smaller steps, staying in a nice, green, running trunk with the rest of the team. I just can't see how we could have done it without knowing the many things we learned through refactoring and exploring and periodic cul-de-sacs in the code. It was bigger than any of our heads.
So what is the point?
- Is the uncertainty the problem, and could we have killed it first?
- An opportunity for links & advice from my small, but wise, readership.
- The merge I'm waiting on is sucking all my CPU and enthusiasm, and I had to do something.
Here's a link that might be relevant...
ReplyDeletehttp://deming-network.org/files/varman.txt
Skip down to "Some basics about variation"
Sounds like your company has some systemic (in the literal academic sense of the word) problems. Asking yourself what you could have differently won't help much. Might even be tampering.
One key I've found is being willing to abandon failed experiments. In preparation for the experiment you branch. You begin teasing out something. After a half an hour you get suspicious. After an hour you know you are doomed but you are trapped in the sunk cost fallacy. After an hour and a half (if you're lucky) you give up. Now is the key moment. Do you say, "Bad job. Throw it away. After a curry and a walk let's run the sequence of changes some other way that retains stability and sanity."? Or do you say, "We're almost there. Let's just leave this branch open for another day/week/month/until I quit"?
ReplyDeleteIn my better moments I am grateful for the learning, discard the experiment, and start over. Looking back, I've always been able to see how to do this. In the moment, I hope I reach 50%.
I have no idea how long you've spent finding this jellyfish. Or how much time you spent doing the changes you're now merging.
ReplyDeleteBut, I do know that if you're in a soul sucking merge, and you know now enough of the territory to belive that the approach you're now using is viable, then you really ought to consider the branch a spike.
Collect all your thoughts as a team about the new vision for this part of the system, the pitfalls of the old one. And start over. One tentacle at a time, on the main branch, one iteration at a time.
There's no sunk cost, the information you've gained the insights that has surfaced are the real value in this case, using them in the most effective way is in my personal oppinion in this context, starting afresh on the trunk. Now you know at least a partial goal, so set the compass and start walking!
I wish I had the clarity and insight to do so more often than I currently do.
Me and Ola Ellnestam have worked effectively with large, multi-iteration refactorings and developed what we called "The Mikado Method" to handle them. Please see http://danielbrolund.wordpress.com/2009/03/28/start-paying-your-technical-debt-the-mikado-method/
ReplyDeletefor a teaser write-up. We see the potential need for this kind of tool and are in the process of writing a book on the matter. Any thoughts on that are welcome.
In Michael Feathers' book I liked the idea of (i think he called it) scratch refactoring: refactoring (possibly without unit tests) with the intent of throwing it away and re-doing it. Maybe it could apply to your situation.
ReplyDeleteI agree with Kent. There have been many times I've abandoned a refactoring attempt that was turning out too big. Sometimes I've pressed on and wasted more time, but I've always been happier when I've tossed it and tried again from a different starting point.
ReplyDeleteGenerally, a different starting point will allow me to whittle away at the edges of the Big Ball Of Mud. (Sometimes it takes several attempts.) The original problem shrinks, and eventually can be vanquished.
At least, that's my experience.
Where's the problem? Just take your time and bill time and material. Legacy Application arise due to a lack of management attention or they're deliberately incurred by a hard cost-benefit analysis which had no room for maintainung the structural or architectural integrity of the software let alone documentation. So there you go.
ReplyDelete