Wallas' Four Stages of Creative Thought

If we want to see a greater theory of productivity, we have to recognize first that software development is thinking.  Once we get there, we need to understand how we can think better. One aid is to have more information so that we know what to do.

In the 1926 book The Art of Thought, Graham Wallas explained that having an idea (a creative solution) requires four distinct phases or steps:

1. Preparation (gathering of theory and data)
2. Incubation (letting the idea "cook" by doing something unrelated)
3. Enlightenment (the emerging of an idea, or "connection")
4. Verification (determining the validity of the idea)

Wallas noted that stages 1 and 4 may take minutes, hours, days, years, or decades. We are primarily interested in those that work on the sub-week scale when programming, but the fact remains that we need to allow the brain to work by feeding it information, giving it some time to process, and then verifying the ideas as they come.

Wallas refers to the mental state of approaching enlightenment as "intimation", which is a very positive and exciting mental state. You know this one: "Oh! Wait! I think I know the answer! It's....." accompanied by bouncing around the room in positive excitement.

Wallas' process is easily recognized by people who work in ideas (software, art, communication, etc).  Many of us have worked hard on one idea for hours or even days only to have it elude us as long as we concentrated on it. We finally surrender, only to have the answer come to us on waking from sleep (sometimes in the middle of the night) or else in the shower, while mowing the grass or exercising, while visiting with family, or while working on something entirely unrelated.

The second step (incubation) is one of the reasons why programming shops need to have ping-pong tables, foosball tables, video games, or a walking course. Without incubation it is hard to reach enlightenment.

Frustratingly, of the four steps, only the final one (verification) looks like "working."  An exceedingly naive manager would make sure that programmers are always typing, not "wasting" their time by reading, researching, or getting "distracted" by games or activities. A foolish manager would make sure developers are spending all their time and overtime concentrating on a single task, thus preventing the crucial incubation stage from occurring.

In a highly collaborative environment, knowledge is shared (preparation) and alternative solutions examined (verification) much more quickly than in solo work. We don't know of any situation where collaboration aids in enlightenment, nor can we say much about incubation other than the usefulness of having someone take a walk with us or play a quick game to distract us from a problem for a few minutes.

If we are wise, we should respect and not disrupt Wallas' four stages of thought.