Releases go out without the drama

The problem

If releases are rare, they become events. Weeks of work gets bundled together, so there's a lot riding on the day.

The run-up is the worst part. "Can this one small thing go in too?" "Is that other bit actually finished?" "If we hold it back, when's the next chance?" Every question is urgent because the window only comes round every few weeks, and every answer is a trade-off between shipping something half ready and waiting another month. Someone picks a quiet evening. There's a checklist. Everyone's on standby.

Then it goes live and something breaks. It always seems to. Not because anyone was careless, but because thirty changes went out at once and nobody can tell which one caused the problem. Undoing it means undoing all thirty, including the ones that were fine.

The lesson people draw is that releases are dangerous, so do them less often. Which makes each one bigger. Which makes it more dangerous. That's the trap, and a lot of companies are stuck in it.

What we do

We do the opposite, and four things make that possible.

Small releases

Releases are small, frequent and boring. A change or two at a time, often several times a day. If something goes wrong, it's obvious what caused it, because only one thing changed, and it's undone in minutes.

There's no scramble to get things into a window, because there's always another release in an hour. Nothing gets rushed in half ready and nothing gets held back for a month.

CI

Every release runs through the full set of automated tests and code checks on a separate machine before it goes anywhere. If anything fails, the release doesn't happen. Nobody can skip this step because they're in a hurry, which is rather the point of automating it.

The same run checks every third-party library the system uses against published lists of known security holes. If one turns up, we find out before it ships, and keeping those libraries patched is part of the ongoing work.

"Is this ready?" stops being a nervous conversation and becomes a green tick.

Staging

A copy of the live system where new work can be seen and tried before it goes live. If a feature isn't what you had in mind, you find out here, and the conversation happens before any customer has seen it.

Feature flags

Some features are big, and they need to reach customers gradually. So they go live switched off, behind a flag. Your own team turns it on for themselves and uses it for real. Then a handful of trusted customers. Then everyone.

If there's a problem at any stage, the flag goes off and the feature disappears in seconds. No rollback, no late night, no undoing thirty other changes.

What it means for you

There's no release day, because every day is release day, and none of them are worth staying late for. New work reaches your customers in days rather than months, and a problem with one change never takes the others down with it.