The development pipeline is a production system

Software developers learn early in their careers that nothing is more urgent than fixing a production outage. Drop everything! All hands on deck!

However, the same level of urgency is not often given to problems with our development tools, build systems, QA environments, and other parts of the software development pipeline. But for the development team, the development pipeline is a production system .

A software developer’s job is to deliver value for the company. Sometimes that means building new features, sometimes that means fixing critical bugs for the customers’ production systems. But none of this can happen when something is broken in the software development pipeline.

If the code can’t compile, the developers are unable to do their jobs, and the team isn’t producing software. For the development team, this is a production outage. Fixing this should be a top priority.

If the QA server is down, the testers are unable to do their jobs, and the team isn’t producing working software. For the QA team, this is a production outage . Fixing it should be a top priority.

In manufacturing, there are extensive processes and procedures on how to prevent and minimize downtime on the assembly line. 1 And similar processes exist for IT service outages . But I’ve found that most of those focus on outages in the service provided to customers , not for the people responsible for building and supporting the services .

I recommend thinking about all the components that take you from “customer wants something” to “that something is delivered to customers”:

A team with a broken development pipeline can’t produce software, and must treat this as a production outage.