Every page we built said noindex, and nothing complained

The dangerous failures in a publishing pipeline are the ones that produce a working site. A build that crashes gets fixed in minutes. A build that succeeds, deploys cleanly, renders perfectly and quietly tells every search engine to forget the domain will run for weeks before anybody notices, and by then the recovery is measured in months.

We came within one command of shipping exactly that.

How it happened

The site can be built for two hosts: the staging hostname while it was being rebuilt, and the production domain. Staging must never be indexed, because two identical copies of the same content competing with each other is the one thing worse than neither of them ranking. So the noindex flag was derived from the hostname, which is the right instinct:

noindex: (import.meta.env.SITE || '').includes('new.sazinga.com') || !import.meta.env.SITE,

Read the second clause. If no site URL is set at all, treat it as unsafe and noindex everything. That is a sensible default in isolation. The problem is the other end of the pipe: npm run build was astro build, with no site specified, so the config fell back to the staging hostname — which means the ordinary, obvious, documented build command produced a site that was noindex on every one of its 245 pages, with canonical tags pointing at staging.

Nothing about that build looks wrong. It completes, the page count is right, the pages render, the content is correct. The only difference is fourteen characters in the head of every document.

What caught it

Not a test. A pre-flight grep, run manually, because the deploy was about to replace a live site and it seemed worth checking what was actually in the files:

grep -o '<meta name="robots"[^>]*>' dist/index.html
<meta name="robots" content="noindex, nofollow">

Every page. Including the homepage.

The fix is trivial once seen — a separate build:prod script that passes --site explicitly, using the CLI flag rather than an environment variable so it behaves identically from cmd, PowerShell and bash. What is not trivial is the class of problem, and we found two more members of it in the same hour.

The second one: a backup file served to the public

Sitting in the public/ directory was robots.txt.dev-bak — a copy of the staging robots file, kept while the production one was written. public/ is published verbatim. So https://sazinga.com/robots.txt.dev-bak was a live, crawlable document whose entire content was:

User-agent: *
Disallow: /

No engine would have honoured it as policy, since it is not at the canonical path. But it is a file on the production domain instructing crawlers to leave, and there is no version of that which is helping.

The third one: a checker that agreed with everything

This is the one that should worry anybody who writes their own tooling. We had built a content audit that walks the built site and reports pages with too little content. Later we taught it something obviously correct — skip pages marked noindex, because a page that cannot rank cannot be measured on whether it would rank well.

Run against a staging build, that tool now reports:

245 pages, 0 below target

A perfect score, produced by a rule that was individually right, applied to a build where every page was noindex. The all-clear meant the exact opposite of what it said.

It now refuses to produce a report at all when every page is noindex, and prints the command to rebuild properly instead. That is the correct shape for the fix: not a warning that can be scrolled past, but a refusal.

The pattern underneath all three

Each of these is a safe default that becomes unsafe when it is the only thing standing between you and production. Noindex-unless-told-otherwise is right for a staging build and catastrophic as a deploy default. Publish-everything-in-public/ is right for assets and wrong for the file you left there while working. Skip-what-cannot-rank is right for a report and wrong when the answer is everything.

None of the three would have thrown an error. All three would have been discovered by the same question: what is actually in the artefact I am about to publish?

What we do now

Three checks run against the built files before anything is uploaded, and a further three run on the server against the staged copy before the swap, any of which aborts the deploy:

• the homepage does not contain noindex
• the canonical resolves to the production origin
• nothing in the build references the staging hostname
• the RSS feed and the IndexNow key file exist

They take under a second and they are all assertions about the artefact rather than about the process that made it. That distinction is the whole lesson. A build script that is supposed to set the right hostname is a belief. A grep for noindex in the file you are about to serve is a fact.