A shared test tenant is global mutable state, and it fails like global mutable state
Three test suites failed on code that had not changed. The failures were in modules with nothing in common, and the error was a seat limit: the tenant had reached its maximum number of members.
Two other suites were the cause. Both needed a second user to test something — one to check that a notification reaches somebody other than the actor, one to check that a permission is refused to a role that lacks it — and both invited a member and never removed it. Every run left one more behind. Nineteen had accumulated, which was enough to reach a seat limit of twenty — a real product rule, correctly enforced, and completely unrelated to anything the failing suites were testing.
Purging the leaked members fixed it in a minute. The fix that mattered was adding seat release to both suites, so that each leaves the tenant with the same number of members it found.
It is global state and it behaves like it
A shared test tenant is a mutable global variable that every suite reads and several write. Every property of global state follows, including the ones people forget when the variable is a database row rather than a symbol:
Failures appear far from their cause. The suites that broke had no relationship to the ones that leaked. Three separate investigations started in the wrong module.
Order matters, and nothing declares it. A suite that passes when run alone can fail after another, and there is no dependency anywhere to inspect.
Effects accumulate. One leaked member is invisible. Nineteen is an outage. The problem is sub-threshold for weeks and then arrives all at once, which is why nobody connects it to the change that introduced it.
The same tenant produced three more incidents in the same period, all of the same shape. A preference controlling whether inventory is tracked had been switched off by other work in flight, which made every stock-receipt suite fail on an unrelated rule. A product soft-deleted by another process left a price-list entry pointing at nothing, so a browser test that picked the newest entry went red mid-session on code that had not moved. And when the tenant’s trial period expired, most of the suite went red at once.
Not one of those was a code regression. All of them cost real time, and all of them were reported first as bugs in whatever module noticed.
Three levels of fix, in increasing order of value
Clean up after yourself. Every suite leaves the tenant as it found it. That is the minimum and it is what the seat leak needed. It is also fragile, because cleanup runs last and last is where a failing test stops.
Restore in a way that survives failure. One suite deliberately expires the tenant’s trial to check that the lifecycle gate refuses writes. Getting that wrong leaves the shared environment locked. It restores the original value in a construct that runs whether or not the test passed — because a cleanup step that only runs on success is a cleanup step that runs exactly when it is not needed.
Do not share. The strongest version is a suite that registers its own tenant, uses it, and never touches anybody else’s. When the shared tenant’s trial expired and most of the suite went red, the ten suites that stayed green were exactly the ten that create their own. That partition is the argument, and it made itself without anybody constructing it.
The lifecycle suite eventually moved to a disposable tenant for a stronger reason than tidiness: it tests transitions — suspended, closed, reactivated — that can leave an environment unusable. A test whose subject is a destructive state transition must never run against state anybody else depends on. The cost of getting it wrong is not a red test, it is everyone else’s afternoon.
Idempotence is the property to aim at
The requirement that made all of this tractable is that a suite must produce the same result run twice in a row without anybody resetting anything.
That is a stronger property than “cleans up”, and it is easier to check. Run it. Run it again. Same result, or there is state leaking somewhere. It catches the leaked member, the left-behind discount scheme, the counter that was consumed, the flag that was toggled — without anybody having to reason about which resources a suite touches.
It also forces a better shape. A suite that must be idempotent stops relying on data it did not create. Seeded fixtures are a slower version of the same problem: a suite that depends on a particular seeded record fails when the seed changes, and the seed always changes.
The one thing that cannot be fixed inside a suite
There is a residue that no amount of suite discipline addresses. When several processes write to one environment concurrently, a run measures the concurrency as much as the code.
A batch run produced two failures that passed on their own. The cause was another process writing to the same tenant while the batch ran. Nothing in either suite was wrong.
The only honest response is to attach conditions to results. A full run made while other work is in flight is useful for noticing obvious breakage and is not evidence. An authoritative run happens when nothing else is touching the environment, and that fact gets recorded next to the number.
The alternative — an isolated database per run — is the right answer and it costs something to build. Until it exists, the cheap mitigation is stating the conditions, because the expensive failure is not a flaky test. It is a team that has learned to re-run red tests until they go green, which is the point at which the suite stops carrying information at all.
Rules
Treat the shared test environment as global mutable state and expect its failure modes. Distant symptoms, order dependence, and effects that accumulate below the threshold of notice.
Require every suite to be idempotent, and check it by running twice. It catches leaks nobody enumerated.
Restore in a construct that runs on failure too. Cleanup that only runs after a pass is cleanup that never runs when it matters.
Give suites their own tenant where the framework allows it. The ones that do survive environmental failures that take everything else down.
Never test a destructive lifecycle transition against shared state.
Record whether a run was contended. A number without its conditions is not a measurement.