Why redesign doesn't happen (enough)

Code degrades because it is strongly anchored to the past.

Instead of updating, refactoring, reorganising, and rearchitecting the code as we learn over the years, a system tends to keep its original shape, as developers shoehorn new features in with If/else statements.

Imagine you are one of N people working individually in the codebase, each of you on a personally assigned task, not in groups.

Any change one of you makes has a non-zero chance of causing a merge conflict for someone else. The larger the change, the greater the chance that some part of it will conflict.

Multiply that by the number of people making changes.

Remember: when separately-made changes conflict, the first person to merge sees no problems, but the second person to merge has to resolve the conflict. 

People hate resolving merge conflicts. It is tedious and exacting work. You have to understand not only your own change, but the other person's, and figure out how to harmonise them. If you get one brace or parenthesis misplaced, however, you'll end up with code that doesn't work.

Now imagine that you've realised that there is a lot of "misplaced semantics" in your code base. There is a data structure, but everwhere it's created there are a set of if statements, and everywhere it is manipulated there are if statements using one member and comparing to a constant... essentially there are clear state-based behaviors, using copied and repeated conditions.

Recognising that this is an open door to a whole class of errors, you convert this to a class, bringing all the if statements into the class, and making the manipulations members of the class, and in short time you've reduced the size and complexity of the codebase, and the data is treated uniformly and correctly the whole way through! Nice work. A nice, well-evidenced refactoring like this is one of the ways a code base grows more coherent over time.

And, of course, you have tests over all the areas of code you changed. These are numerous and fast-running, all integrated into the test suite and all passing.

You check the changes in...

What are the chances that some change by some person touched any of the places you just improved? Or any of the tests that were revised? 

The thought of disrupting your N-1 is enough to stop all forward movement.

The change is valid, good, proven, and by golly it makes the program safer to change and easier to read than ever before! You don't even need 1/3 of the comments that were there before. This is a righteous change.

It also touched 15 of production code files (further proof it was needed: those semantics had already spread that far!) and many dozens of lines.

Now you think of the N-1 colleagues, any or all of whom may have a PR already waiting or code changes that simply started before your nice change.

Some of them will slow down and might even introduce a bug while resolving the merge conflict.

If they are slowed down, the manager is probably going to be aware of it, and certainly they will be if a defect results. You'll take the blame for the defect and for the slowdown.

Here is your moment of decision. Do you push the change, or abandon it?

This fear of changing code is a significant social effect.

The code remains anchored to some point in the past, it's original design ossified.

Pretty soon all of the functions the software ACTUALLY performs are written in as EXCEPTIONS to the flow it no longer follows.

What interventions would defang this fear and allow teams to move the design and architecture forward?

That question is the point of this post. What do you think? How would you change your ways of work?