A redirect that returns 404 teaches a search engine nothing

11 September 2026

A site migration is judged on its redirects. Not on the design, not on the copy, and not on how much faster the new thing is. If the addresses the world already knows do not resolve to the right places, the new site is a second site that happens to share a domain with the old one.

We knew that, wrote the redirect map before cutover, generated it from the old site’s own URL inventory joined against a hundred and eighty days of Search Console data so that the busiest addresses were covered first, and asserted that every target returned 200 before writing anything. A hundred and forty entries. Then we cut over, and for days afterwards a site: query on Bing returned a page from the old marketing site, with its old copy attached, as though nothing had happened.

The symptom looked like a content problem

It is easy to read a stale search result as a crawling problem, or as an indexing delay, or as the engine being slow. All three are plausible and all three would have led us to wait. What actually settled it was checking the thing being asked for rather than the thing being returned.

The indexed URL was /digital-solutions. Our map had /digital-solutions/.

Why one character mattered

The site is built with trailingSlash: 'always', so every real page lives at a path ending in a slash. When a browser asks for /services, nginx notices that a directory called services exists on disk, and issues a 301 to /services/. That behaviour is automatic, it is why the no-slash form of every live page works, and it is why we never noticed the gap.

A retired WordPress path has no directory. There is nothing on disk called digital-solutions. So nginx had no reason to add a slash, the request never matched the map — which was keyed on the slashed form — and it fell through to a real 404.

That is the whole bug. Every entry in the map was generated with a trailing slash because that is how the old site’s canonical URLs were written. Search engines, however, hold whatever form they first encountered, which for a decade of inbound links, directory listings and pasted addresses includes plenty without one.

A 404 is not a neutral answer

This is the part worth internalising. We tend to think of a 404 as the absence of information. For a crawler it is information, and it is the wrong information: it says the address is dead, not that it moved. So the engine keeps what it already has — the title, the description, the cached snippet — and re-queues the URL to try again later. The stale result survives precisely because the server said nothing useful.

A 301 is the opposite. It transfers the identity of the old address to the new one, and it is the only mechanism by which the equity built up over years of the old site accrues to the new pages rather than evaporating.

The access log shows the transition in a single view. On one URL, eighteen responses of 404, then ten of 301. Same crawler, same address, different answer, because in between we changed what the server said.

The second bug in the same map

While fixing the slash problem we found that the map was keyed on $request_uri, which includes the query string. Any mapped URL arriving with ?utm_source=... on the end — which is to say, most of the ones that came from anywhere other than a plain link — did not match either.

Both problems have the same fix. Rather than doubling the map to hold a slashed and unslashed form of every entry, normalise the key once before the lookup:

map $uri $legacy_key {
    default              "${uri}/";
    "~\.[A-Za-z0-9]+$"   $uri;   # a file — leave it alone
    "~/$"                $uri;   # already ends in a slash
}

$uri carries no query string, so that problem disappears as a side effect. The regex exception for anything with a file extension matters more than it looks: without it /sitemap_index.xml becomes /sitemap_index.xml/ and stops matching.

The one that mattered most was a sitemap

Among the URLs still being requested and still returning 404 was /sitemap_index.xml — the path Yoast uses, and therefore the path both Google and Bing had on file for this domain. Our new sitemap is at /sitemap-index.xml. A hyphen instead of an underscore.

An engine asking for a sitemap is asking to be told what exists. Answering 404 to that specific request, for days, while wondering why discovery is slow, is a self-inflicted wound.

What we would do differently

Generate the map, then test it the way an engine would rather than the way you wrote it. Ours now has a check that reads the map off the server and requests every entry three ways — as written, without the trailing slash, and with a query string appended — asserting a 301 to the expected target each time. On the current map that is 141 entries and 420 requests, it runs in seconds, and it would have caught this before cutover rather than a week after it.

The broader rule: after a migration, read the access log for 404s grouped by user agent. The crawlers will tell you exactly which addresses they still believe in. Ours were asking for /digital-solutions, /technologies, /start-hiring, /home, the category pages, and the old RSS feed at /feed — and each one of those was a link somebody had built to us, being answered with nothing.

They are the cheapest SEO work available, because the demand already exists. You are not trying to earn attention. You are trying to stop throwing away attention you already have.