Search engines rank pages; answer engines quote them. When ChatGPT, Perplexity or an AI browser agent answers a question, it fetches a handful of pages, cuts them into passages and cites the passages that survive the trip. A page that renders beautifully for humans loses at this game in three separate places, and none of them show up in a normal analytics dashboard.
Access: the fetch that never happens
Every citation starts with a plain HTTP GET, and robots.txt decides whether that GET is welcome. The file format dates back to 1994 and was only standardised in September 2022 as RFC 9309. Three failure modes dominate in practice: a robots.txt that returns a 5xx error (RFC 9309 tells crawlers to treat the whole site as disallowed), a blanket rule written years ago that now blocks retrieval bots along with scrapers, and firewall rules that serve challenge pages to any non-browser user agent.
The fix costs one afternoon: serve robots.txt with a 200 or a 404, never a 500; list the agent classes you accept explicitly; and test the page with a plain GET from outside your network. A blocked fetch is worth 0 citations no matter how good the content is.
Parseability: server HTML versus a JavaScript shell
Retrieval systems work with the HTML the server sends, and only some of them run JavaScript afterwards. Google documents its two-wave approach in its JavaScript SEO guide, but most answer-engine fetchers are single-wave: no render, no hydration, no client router. A single-page app that ships an empty <div id="root"> ships exactly nothing to those systems.
Measure the gap directly: fetch your page with curl, strip the tags and count the sentences that remain. Lekta grades this layer as Indexability and expresses the same measurement as a JS dependency coefficient between 0 and 1 — the share of visible text that exists only after rendering. Pages under 0.01 are effectively static; pages above 0.5 gamble their entire content on a render pass that most fetchers never run.
Answerability: content shaped for quotation
A fetchable, parseable page still competes on whether its passages answer questions. Concrete data wins: dates, numbers, names and units give a language model something to quote and a fact-checker something to verify. Structured markup helps the same way — an Article object with datePublished and dateModified tells the engine how fresh the page is, and an og:type declaration tells it what kind of document it is reading.
Freshness must be visible as well as machine-readable. A dateModified stamp in JSON-LD with no date anywhere on the rendered page reads as decoration, and several engines discount it. Print the date where a human sees it, keep the markup in sync, and update both when the content actually changes.
Where to start on a real site
Audit one representative article page, not the homepage. Landing pages are navigation; articles are what answer engines quote. Check the three layers in order — a robots.txt problem invalidates everything downstream, so fixing layers out of order wastes work. The whole loop for one page takes under an hour, and the access and parseability fixes usually carry over to the entire site at once. Lekta grades a fourth layer, Recency: the visible date the answerability section already told you to print.
Sources
All four references are primary: the RFC text itself, the vocabulary definitions at schema.org and ogp.me, and Google’s own crawler documentation. Read them in that order and they map one-to-one onto the three layers above.