Fix Crawlability Before Content

Layer one, crawlable: are retrieval crawlers allowed, does a CDN or edge layer block outside your site config, does key content depend on client-side rendering. The evidence is in server logs.

Layer two, retrieved: does the sitemap declare the right domain and every indexable page, does structured data appear in server-rendered HTML, are same-intent pages competing.

Layer three, mentioned and cited: mention rate, average position and cited-source composition across a fixed question set.

Layer four, converting: does visibility produce visits, do landing pages answer the intent, is the enquiry path intact.

If crawling fails, no amount of rewriting reaches the crawler and the investment returns zero.

Yield in this category also concentrates in the top ten — around 13 monthly clicks per keyword at positions 1–3 and near zero beyond 11 — so the gap between fixing the foundation first and fixing it later gets amplified by position bands.

Teams tend to start with content because content is visible output. Crawl-layer faults are not visible on the page — it renders perfectly in a browser, and only the crawler receives a shell.

So the first step is pulling server logs and reading request volume, status codes and response sizes by user agent. It costs very little and rules out the most expensive class of error.