If One Small Change Breaks Half Your Tests, Your Tests Are the Problem

You make a small change. A form field gets renamed. A new column is added to a table. Nothing that affects how the business works.

You run the test suite. Two hundred tests fail.

We've all been there. And I'll be honest about what often happens next, because I've done it: some of those tests get marked as skipped "just for now". I've had tests sitting ignored for over a year.

An ignored test protects you from nothing. It's worse than no test, because it looks like protection.

The problem wasn't the change. It was that the tests were tightly coupled to the code they were testing. This article is about fixing that.

What a test suite is actually worth

Here's a simple way to think about the value of tests:

Value of tests = cost of the bugs they find − cost of the test suite

We usually focus on the first half. More tests, more coverage, more bugs caught.

The second half gets forgotten. Tests cost money to write. They cost money every time a change breaks them and someone has to fix them. They cost time every time the suite runs.

Cut that cost and the same tests become more valuable. And there's a bonus: a suite that doesn't break every time you change something makes the whole codebase easier to change.

The biggest driver of test suite cost is coupling.

Coupling, quickly

Coupling is how much one piece of code knows about the inner workings of another.

Low coupling: your application tells an email gateway "send this message to this address". It has no idea whether that happens over SMTP, via an API, or through a queue.

High coupling: your application knows the transport, the credentials, the retry logic. Change any of those and the change ripples through the whole system.

We work hard to keep production code loosely coupled. We use interfaces, dependency injection, layers.

Then we write tests that know everything about everything.

I'm going to go through three stories from a real-ish project, each showing a different kind of test coupling and how to fix it. The project is a workplace quiz app. Users belong to teams. They answer multiple choice quizzes. Scores go to the team the user was in at the time, and teams compete for prizes.

Story 1: UI tests that know too much

The first version of the tests drove the application through the browser. Something like this:

public function testTeamScore(): void
{
    $this->visit('/');
    $this->click('Log in');
    $this->fillField('username', 'bob');
    $this->fillField('password', 'password');
    $this->press('Submit');
    $this->click('Quiz 1');
    $this->selectOption('question_1', 'a');
    $this->selectOption('question_2', 'b');
    $this->press('Submit answers');
    // ... log in as admin, find the team page, read the score ...
    $this->assertSame('7', $this->getText('#team-score'));
}

Two problems.

You can't tell what's being tested. Is this about logging in? The quiz form? Team scores? The intent is buried in clicks.

Every test knows about every page. Rename the username field and every test that logs in fails. Change how quizzes are listed and every test that takes a quiz fails. Hundreds of tests, all coupled to the same fragile details.

Fix 1: page objects

The first improvement is a well known pattern. A page object wraps a page (or part of one) and gives tests a clean interface to it.

A page object does three kinds of thing: it does what a human would do (fill in a form, click a button), it reads data from the page, and it navigates to other pages.

$loginPage = $homePage->goToLoginPage();
$myQuizzesPage = $loginPage->logIn('bob', 'password');
$quizPage = $myQuizzesPage->openQuiz(1);
$quizPage->answerQuestion(1, 'a');
$quizPage->answerQuestion(2, 'b');
$resultsPage = $quizPage->submitAnswers();

$this->assertSame(3, $resultsPage->getScore());

Now, if the username field is renamed, you fix LoginPage once, and every test works again.

It's much better. But it's still not great. What if the flow after logging in changes? Say users now land on a dashboard instead of their quiz list. Every test that logs in and goes to a quiz still has to change, because the navigation is in the tests.

Fix 2: a domain language layer

The next step is to add a layer above the page objects that speaks the language of the business, not the language of web pages:

public function testIndividualScoreIsAllocatedToTheirTeam(): void
{
    $this->assignUserToTeam($bob, $teamApple);

    $this->submitUsersAnswers($bob, self::QUIZ_1, [
        'engagement' => 'a',
        'enjoyment' => 'b',
    ]);

    $this->assertSame(7, $this->getTeamScore($teamApple));
}

Read that test out loud. It is the business requirement. "When Bob answers a quiz, his score goes to his team."

The stack now looks like this:

Tests   →   Domain language layer   →   Page objects   →   Application

Each layer has one job. Tests say what should happen. The domain language layer knows which pages to use. The page objects know how each page works.

Change the login flow and you update the domain language layer. Change a form and you update a page object. The tests only change when a business requirement changes.

That's the goal. A test should break when the behaviour it tests changes, and not otherwise.

Story 2: testing business logic through the UI

Even with all those layers, UI tests have a fundamental problem. They test your business logic through the most fragile, slowest and most frequently changing part of the system.

Think about the test pyramid. Unit tests at the bottom: fast, cheap, stable. UI tests at the top: slow, expensive, brittle. When I talk about UI tests here, I mean testing the application's functionality through the UI, not testing the UI itself.

Most of the questions we care about, like "does Bob's score go to his team?", have nothing to do with the UI. So why test them through it?

Where should business logic tests go?

If you've come across layered or hexagonal architecture, this will look familiar.

Business logic sits at the centre. Around it is a service layer, expressing what the application can do in business terms. Outside that is the framework, dealing with HTTP, databases, email and the rest of the messy real world.

The golden rule: inner layers know nothing about outer layers. Your business logic has no idea it's part of a web application.

The service layer is where business logic tests belong:

interface AnswerSubmissionService
{
    /** @param array<string, string> $answers */
    public function submitUsersAnswers(User $user, int $quizId, array $answers): void;
}

Look at that method name, then look back at the domain language layer from story 1. They're almost identical. The service layer already speaks the language of the business. The layer between your tests and the application gets very thin, possibly thin enough to remove.

And the service layer changes far less often than the UI. Designers redo pages. Business rules about how scores are allocated change much less.

Gateways for the outside world

What about side effects? Later in the project, users had to pay before entering some quizzes, and results were emailed out. You don't want tests sending real emails or taking real payments.

The business logic depends on interfaces, not implementations:

interface EmailGateway
{
    public function send(string $to, string $subject, string $body): void;
}

In production, an adapter in the framework layer implements it using your real email provider. In tests, you swap in a spy that just records what was sent:

final class EmailGatewaySpy implements EmailGateway
{
    /** @var list<array{to: string, subject: string, body: string}> */
    private array $sent = [];

    public function send(string $to, string $subject, string $body): void
    {
        $this->sent[] = ['to' => $to, 'subject' => $subject, 'body' => $body];
    }

    /** @return list<array{to: string, subject: string, body: string}> */
    public function getSentEmails(): array
    {
        return $this->sent;
    }
}

Now your test can call the service layer, then check exactly which emails would have been sent. No real email, no real payment, and a test that runs in milliseconds instead of seconds.

The thought experiment

Here's a question I like to ask. What if the business decides to replace the website with a mobile app?

If all your business logic tests go through the web UI, they're all useless. You're starting again.

If they go through the service layer, they all still work. Every rule about scores, teams, payments and emails is still tested. You only need new tests for the new front end.

So do you still need UI tests?

Some, yes. But the question they answer is much smaller.

Once the service layer is well tested, the only question left for the UI is: "is it wired up to the service layer correctly?" That needs far fewer tests. Take the minimum number of scenarios needed to be confident the wiring works, and test those through the UI.

Whether to automate even those depends on the project. If the UI changes constantly, automated UI tests will be expensive to maintain. If the UI is small and stable, and most of the system is below the waterline, a quick manual check before release might be cheaper. There's no universal answer. It's a cost decision.

What if your code isn't built like this?

Most legacy code isn't. Business logic lives in controllers. Email is sent from the middle of a form handler.

You can get there gradually:

1. Keep the UI tests for now. They're your safety net.
2. For each external service (email, payments, and so on), introduce a gateway interface and an adapter, and change the code to use it.
3. Move business logic out of controllers and behind a service layer.
4. Write service layer tests for the logic you've moved.
5. Once the service layer tests give you confidence, retire the UI tests that duplicate them.

It's not quick. But every step makes the codebase better as well as the tests.

Story 3: test data that knows too much

The third story is the one that caught me out most recently.

The quiz app went multi-tenant. Each company got its own subdomain, and every user now belonged to a company. That meant a new company_id column on the users table.

A sensible, contained change. And yet tests that had been stable for months, like "does an individual's score get allocated to their team?", started failing all over the place.

The reason: test data was loaded straight into the database from YAML files. None of those users had a company_id. Every test that needed a user was broken.

Seeding the database directly couples your tests to your database schema. The schema is an implementation detail. Your tests about team scores shouldn't care about it. You might not even be running against a real database in tests, if you're using in-memory fakes for speed.

Building objects inline in tests has the same problem in a different form:

$company = new Company('Acme');
$user = new User('Anna', 'password', $company, $teamApple);

Add a new constructor argument and you're editing every test that creates a user.

Fix: object mothers and test builders

Two patterns help here.

An object mother gives you domain objects in known, valid states. I like to name them after the personas from user stories:

$anna = $this->users->anna();

Behind the scenes, the object mother creates Anna (using the real application code to register her, not raw SQL) or returns her if she already exists. If Anna needs a company, the user object mother asks the company object mother for one.

That last part is the important bit. When users gained a company, only the user object mother needed to change. It started asking for a company. Every test that just wanted "a user" carried on working without modification.

A test builder is for when you need something a bit different. It has sensible defaults for everything, and you only override what your test cares about:

$user = (new UserBuilder())
    ->withName('Annabelle')
    ->inTeam($teamBanana)
    ->build();

Here's a minimal one:

final class UserBuilder
{
    private string $name = 'Test User';
    private string $password = 'Passw0rd!';
    private ?Company $company = null;
    private ?Team $team = null;

    public function withName(string $name): self
    {
        $clone = clone $this;
        $clone->name = $name;
        return $clone;
    }

    public function inTeam(Team $team): self
    {
        $clone = clone $this;
        $clone->team = $team;
        return $clone;
    }

    public function build(): User
    {
        return new User(
            $this->name,
            $this->password,
            $this->company ?? (new CompanyBuilder())->build(),
            $this->team ?? (new TeamBuilder())->build(),
        );
    }
}

The test says only what matters to it: a user called Annabelle in team Banana. When the User constructor changes, you update build() in one place.

If you're attached to YAML files for your fixture data, that's fine. Keep the data in YAML if you like, but load it through builders or your application's services, not straight into the database.

The common thread

Look back at the three stories and the same idea runs through all of them.

We already know how to write loosely coupled production code. Interfaces. Layers. Single responsibility. Don't repeat yourself.

Test code deserves the same care. It's code. It needs maintaining. It benefits from the same disciplines:

• Page objects and a domain language layer decouple tests from the UI.
• A service layer and gateways decouple business logic tests from the UI and the outside world.
• Object mothers and builders decouple tests from how objects are constructed and stored.

The test for whether you've got it right is simple. When a test fails, is it because a business requirement changed, or because an implementation detail changed? If it's the second, there's coupling to remove.

Warning signs

How do you know if your tests are too tightly coupled? Some signs to look out for:

Tests are hard to read. If you can't tell what a test is checking without reading every line, it's probably full of implementation detail that belongs somewhere else.

The same setup code appears in lots of tests. Every copy is another place to update when that setup changes. That's a builder or an object mother waiting to be written.

Tests mention things the business doesn't care about. CSS selectors, database column names, HTTP status codes in tests that are supposed to be about quiz scores. Each of those is a coupling point.

People are nervous about refactoring. If developers avoid improving code because "it'll break loads of tests", the tests have stopped being a safety net and become a cage.

Skipped tests are piling up. Check how many tests in your suite are currently marked as skipped or incomplete, and how long they've been that way. It's often an uncomfortable number.

The suite is slow. Not always a coupling problem, but it's often a sign that business logic is being tested through the UI or a real database when it doesn't need to be.

Where to start

Find the last change that broke a lot of tests. Ask why it broke them.

If it was a UI change, introduce page objects. If it was a schema change, introduce a builder or object mother for that entity. If your business logic can only be tested through the browser, pick one feature and pull its logic behind a service interface.

You don't have to fix everything at once. Each small piece of decoupling makes the next change a little cheaper.

The full talk from the Dutch PHP Conference goes through each of these stories in more depth.

What's the most tests you've seen break from a single, innocent change?