Progressive Admission - pattern for data processing

 I don't know if there is another name for this pattern, but I've been calling it "progressive admission" and using it for many years.

It is a story splitting skill, related to the "walking skelton" AKA "tracer bullet" technique.

Say you are about to build a program that reads from a message queue and processes messages. 

Start Closed

The first stage is to build the end-to-end system (reader, checker) that does nothing. No matter what you feed it, the front-end checker (correctly) reports that it's not implemented. This is beginning all closed. 

At this stage, you can start testing the clients and the exception handling/reporting. 

First Admissions

Now you pick the simplest message. It could be a health message or an ill-formed message. This one message or message type is admitted. The program correctly rejects the ill-formed message, or answers the health message with a hard-coded message result.

At this time, all other messages are rejected by default. The chosen message is the only exception. It is "admitted" to the processing side.

The system is not only unit-testable (ensure it rejects ill-formed messages) but it is end-to-end testable and could be harmlessly deployed. 

Additional Admissions

Continue handling specific messages while all others are rejected as "not implemented" still.

Eventually you will be set to handle the simplest, or most common, or most interesting message. You will admit exactly one kind of message that slips past the checker into the processing section. Instead of rejecting it as unimplemented, you build a simple implementation.

This one kind of message is truly implemented. You have widened the admission by one message type or shape.  Any message that doesn't come in that shape is rejected as invalid or not implemented.

You can likely admit one message at a time and possibly cover several iterations in a day. Each is well tested (unit and end-to-end). You can TDD these, refactoring after every green state.

"Final" Admission

At some point, you will have implemented the last of the planned messages, and you've done so with full testing, and any message that "surprises" the service is rejected according to a well-established pattern.

The service is probably already deployed and likely has been doing useful work for some time. You didn't have to wait until you covered all the cases and all the corner cases -- this was automatically done with sharp focus.

Continuing Admissions

Software tends to never be finished as long as people are using it. Each change to a message and every new message or schema can be implemented fairly easily using progressive admission for the various shapes and conditions that apply.

Versioned Interfaces

It's wise to version your interfaces. When you add a message, you bump the minor version. Only when an existing message is changed or deprecated will you bump the major. Only changes to existing messages are "breaking" changes.

Applicability

It's not just message-processing applications. It's anything with a clear input-processing-output cycle. This is easier to imagine with a microservice that reads a queue, but it can be a batch file input, or http operations, or just about anything, including input fields and UI controls.

I'm not saying that you can, must, or should do everything this way, but it's a pattern worth considering.

AKA

I don't know if someone already named this - I'd be shocked if nobody did. It's a handy way to slice story implementation, though. As I find other names, I'll list them here with links.