Reading Search Console past your pages.

Search Console sent an alert: new reasons prevent pages from being indexed — redirects, 404s, server errors. The instinct is to audit your pages. We audited ours and found them clean, which was the useful result: every one of those three rows was about something other than the pages we publish.

The 404s were a security feature talking to a crawler

The single largest 404 source — over a hundred and sixty hits — was a path we never created: our CDN's email-obfuscation feature rewrites the contact address in our footer into an encoded link under its own path, and crawlers that follow it get a 404. A protection designed for human browsers was manufacturing crawl errors at machine scale. One settings toggle ended it; the email is public on purpose anyway.

The rest of the 404 volume was vulnerability scanners probing for WordPress installers and exposed configuration files on a site that has neither. That noise is the ambient weather of the public internet, and treating it as a to-do list is a category error.

The server errors were not the website at all

The 5xx row traced to a subdomain running a private analytics dashboard, whose origin machine times out when it is off. A domain-level Search Console property counts every subdomain, so a personal tool's downtime was landing in the same report as the public site — which stays comfortably at zero server errors on its own.

The fix respected what the subdomain is: private. Not a DNS change, not a forced uptime — an edge rule that blocks verified crawlers from that hostname entirely. Search engines have no business indexing a private dashboard, so keeping them out solves the report noise and the exposure in one move.

The redirects were one habit, expressed three ways

The redirect row was ours, and instructive. Our host serves pages at trailing-slash paths and redirects the slashless form. Our internal links used the slashless form — in the footer's HTML, in the header's JavaScript link objects, and in a generated machine-readable index. Every internal click paid a redirect, and a few canonicals pointed at redirecting URLs.

Fixing it took three passes, because the same habit hid in three syntaxes, and the audit regex that caught HTML attributes walked straight past JavaScript object literals. The lesson we kept: verify fixes against the live site with cache-busting requests, because the second and third instances only surfaced when the "fixed" pages were fetched fresh and still carried old links.

The triage order that made this fast

What turned a vague alert into three closed cases in an evening was refusing to start from the report. We started from the edge: the CDN logs every request with status, path, and requesting agent, so the question "which URLs are actually failing, for whom" has an exact answer — while the indexing report only says that failures exist somewhere in the property.

Edge log first, own-crawl second — walk your sitemap and every internal link the way a crawler would, checking each response without following redirects. Only then read the report, as a summary to reconcile rather than a mystery to interpret. Everything in it should map to something you already found; whatever does not is the actual surprise worth investigating.

What an indexing report is actually for

A domain property's error rows aggregate your pages, your infrastructure's side effects, your private subdomains, and the internet's background probing into one undifferentiated list. Read as a verdict on your content, the list produces panic and wasted audits. Read as a prompt to check the edge, it is a decent smoke alarm — it told us where smoke existed, and the logs told us that two of the three fires were not in our house.

← All notes