Nov 232013

How do you blow half a billion dollars on a national Web site that can’t complete a thousand enrollments a day for weeks after launch? Use the “waterfall” approach–an outdated, top-down management style that is a bureaucrat’s dream and a user’s nightmare.

The waterfall style of project management sounds so natural that it hasn’t even needed a moniker until now: think of an idea, plan its design, build it, and only then deploy to your users. But there is an alternative. In recent years it’s come to be called agile programming, though its roots lie in open-source development (which some historians argue dates back to Henry Ford’s breaking of the patent on 2-cycle gas engines).

Perhaps the most historically influential screed against the waterfall is Eric Raymond’s essay “The Cathedral and the Bazaar,” whose most famous maxim is:

Release early. Release often. And listen to your customers.

Without invoking Raymond or open-source by name, Internet observer Clay Shirky pins the blame on the waterfall.

The problem with was not timeline or budget. The problem was that the site did not work, and the administration decided to launch it anyway.

This is not just a hiring problem, or a procurement problem. This is a management problem, and a cultural problem. The preferred method for implementing large technology projects in Washington is to write the plans up front, break them into increasingly detailed specifications, then build what the specifications call for. It’s often called the waterfall method, because on a timeline the project cascades from planning, at the top left of the chart, down to implementation, on the bottom right.

healthcare_gov_crashLike all organizational models, waterfall is mainly a theory of collaboration. By putting the most serious planning at the beginning, with subsequent work derived from the plan, the waterfall method amounts to a pledge by all parties not to learn anything while doing the actual work. Instead, waterfall insists that the participants will understand best how things should work before accumulating any real-world experience, and that planners will always know more than workers.

This is a perfect fit for a culture that communicates in the deontic language of legislation. It is also a dreadful way to make new technology. If there is no room for learning by doing, early mistakes will resist correction. If the people with real technical knowledge can’t deliver bad news up the chain, potential failures get embedded rather than uprooted as the work goes on.

The key phrase is “a pledge by all parties not to learn anything while doing the actual work.” The Hollywood scripts that float through our collective consciousness don’t help at a time like this.

Intoning “Failure is not an option” will be at best useless, and at worst harmful. There is no “Suddenly Go Faster” button, no way you can throw in money or additional developers as a late-stage accelerant; money is not directly tradable for either quality or speed, and adding more programmers to a late project makes it later. You can slip deadlines, reduce features, or, as a last resort, just launch and see what breaks.

How can you spot a waterfall in the making and turn around before you fall over it? Just look at its timeline, as demonstrated in this infographic on how to tell a promising project from a doomed one. If your new media project follows steps (first conceive, then design, then implement) instead of stages (version 1.0, 1.1, 1.2) then you’re doomed to fail.

To give a golf metaphor: no one would ever plan to sink a par-5 hole by dividing the distance by five and hitting five equidistant shots in its direction. There are just too many unpredictable variables between you and the final putt. Instead, you try to get as close to the hole as possible with your first shot, knowing you’ll be way off but understanding that you’ll compensate in your subsequent shots. Aiming for the hole in stages rather than steps gives you a much better chance of sinking the ball under par.

golf_ball_holeEven if the game gets called for rain halfway through, you’ll still be at a closer stage than if you had expected all your steps to proceed as planned. Regardless of the weather, correcting as you go will make it much more likely your ball will reach its destination–and avoid getting carried down the waterfall.