A new website can look perfect and be invisible to Google

You launch the new website. You click through it and every page is right: the words, the photographs, the contact form. Then a few weeks later nobody is finding you, and the people who built it tell you everything is working.

It was working. That is the trouble. A website has a reader you never meet, a search engine’s program, which arrives once, takes what the server gives it and leaves. It does not log in, it does not complain, and it does not come back because a page looked odd. If it is told the wrong thing, you find out weeks later from a slow decline, and recovering takes months.

We build booking systems, compliance registers and pricing tools, and we rebuilt our own website over a fortnight. Our own build nearly did this to us. This is what we found, because the habits that are right for business software are wrong for a website, and the people who build one need to know the difference.

What was actually going on

Our site is built for two addresses: a private staging address while we worked, and the real domain. Staging must never appear in search, so the build marked every page “do not index” whenever it thought it was building for staging. That is sensible. But the ordinary build command did not say which address it was building for, so it fell back to staging. The result was a site with “do not index” on all 245 pages, and every page pointing at the wrong address as its proper home.

That build finished without an error. The page count was right and every page displayed correctly. Put live, it would have looked healthy while telling every search engine to forget the domain, and no dashboard would have said so. Behind a login, a wrong setting breaks a feature within the hour. Here the feedback takes weeks.

The second failure was quieter. We had a map of 140 old addresses from the previous site, each pointing to its new home, and we checked that every target existed. Then we served the old content anyway, for days. The map was written with a trailing slash on each address, as the old site wrote them, but search engines hold whatever form they first saw, and many of ours had no slash. Live pages worked either way, which is why nothing looked wrong. The retired addresses fell through to “not found”, and a “not found” tells the search engine the page is dead, so it keeps the stale entry it already has. The symptom was a search result showing marketing copy for a company we had stopped being.

The third was the library of articles. We had 142, and only 11 linked to another article. A reader who finished one had nowhere to go, and the crawler had almost no structure to follow.

What we changed

The deploy now checks the finished files, not the process that made them. The home page must not contain “do not index”. The proper address must be the real domain. Nothing may mention the staging address. The feed and key files must exist, and the page count must be over 200. Each check exists because of something that went wrong, and each takes a second.

For the old addresses, a check reads the map from the server and requests every entry three ways: as written, without the trailing slash, and with extra text on the end. It expects a permanent redirect each time. That is 141 entries and 420 requests, and it runs in seconds. The map had never been tested before.

For the articles, every one now carries three related posts, 408 links, worked out from the text at build time. The first version was wrong, because a shared topic alone cleared the bar and with 79 articles under one topic the suggestions were close to random. The second matched an article about payment duplicates to one about steel grades because both used the word “quotation”. The third weights distinctive words above common ones.

What it did not fix

None of this tells you whether anyone found the site. Search Console, the free report from Google, lags by two to three days and tells you whether a page was crawled, which is different from whether it was kept. The web server’s access log shows which visitors arrived and which old addresses they still believe in.

The only direct measure is asking the answer engines the question a buyer would ask. On the same day, with the same access, one engine described our product accurately from the new pages and another said it had not enough public evidence to identify it. The difference was not access. It was whether a page held a sentence specific enough to quote.

The pattern, for anyone buying a website

Ask the people building it four things. What is in the finished files before you upload them: can they show you the page head, not a screenshot? What happens to every old address, and is that tested or hoped? How will you know, within a week rather than a quarter, that search engines can see the site? And who checks the staging copy cannot be mistaken for the real one?

Every failure above had an ordinary fix: a build setting, a status code, a test. None could be found by looking at the site, and none was about the words on the page. If your builders do not think in those terms, the site will look correct and quietly do nothing, and nobody will be able to tell you why.

Where this ends up

The same habit of checking the output, not the process, is what goes into the booking systems, registers and pricing tools described under custom software development, applied there to people who do log in.