Changes go in without breaking anything

The problem

You ask for a small change. Something to do with invoicing, say. It goes in, it works, and three days later a customer tells you the search page has stopped working. Nobody can see how the two are connected, but they are.

After this has happened a few times, a pattern sets in. Every change becomes a gamble, so changes get batched up, delayed, or quietly dropped. The software stops keeping up with the business, not because anyone decided it should, but because asking for changes has become too risky.

The underlying cause is that nobody can say with confidence what the system is supposed to do. So nobody can check that it still does it.

What we do

We build that confidence in two layers.

Automated tests

For every piece of behaviour in the system there's a test: a small program that exercises that behaviour and checks the result. Create an order, check the total is right. Log in with the wrong password, check you're kept out. Thousands of these, covering everything the system does.

Every time a developer changes anything, all of them run. If the invoicing change has broken the search page, a test fails within minutes, on the developer's machine, before the change has gone anywhere near a customer.

This changes how change feels. The question "will this break anything?" has an answer, and the answer comes from a machine rather than from someone's memory of how the system fits together.

Not every test does the same job. Some check a single calculation on its own and run in a fraction of a second. Some check that the pieces fit together, like the code and the database it talks to. A few drive the whole system the way a user would. Each kind catches problems the others miss.

We don't aim for a coverage percentage. A figure like "80% covered" tells you how much code the tests ran, not whether they'd notice if it broke. The aim is a set of tests you can trust when you make a change. And test code is held to the same standard as everything else, because a test nobody can read is a test nobody will fix.

Code checks

Tests check what the system does. Code checks read the code itself and look for the kinds of mistake that lead to bugs, before the code has even run. Think of a very pedantic reviewer who reads every line, never gets tired, and never lets anything slide because it's Friday.

Some of these checks are standard and come with the tools we use. The more valuable ones are the rules we write ourselves. When a bug gets through, we ask whether we could have caught it automatically. Usually the answer is yes, so we write a rule. That kind of bug is now impossible to write again, on this project and often on every other project we work on.

Rules can also enforce decisions about how the system fits together. Say one part of the code must never talk to the database directly. Instead of writing that in a document and hoping everyone reads it, we write a check, and a change that breaks the rule doesn't get in. That matters most for new developers, who can't know a rule nobody has told them.

On a new project, the checks are as strict as they go from the first day. An existing codebase is different. Switch everything on at once and you'd get thousands of warnings about code that has worked fine for years. So we record what's already there as a baseline and hold every new change to the full standard. The code gets better with every change, without a big cleanup project. We wrote SARB, the open source tool that does this for PHP.

Over time the system gets harder and harder to break, because it's accumulating rules from every mistake anyone has made.

What it means for you

You can ask for changes without bracing for the fallout. Small changes go in quickly and safely, which means the software keeps pace with the business rather than falling behind it.

And when we say something works, that's because it's been checked, every time, by something that doesn't forget.