Learn · guide4 min read

Learn / Machine readability

Machine readability: what answer engines actually need from your page

Answer engines cite pages they can fetch, parse and quote. A tour of the four layers where pages silently fail: access, indexability, answerability and recency.

PublishedUpdatedCited basis
JavaScript SEOIndexability layer

Search engines rank pages; answer engines quote them. When ChatGPT, Perplexity or an AI browser agent answers a question, it fetches a handful of pages, cuts them into passages and cites the passages that survive the trip. A page that renders beautifully for humans loses at this game in four separate places, and none of them show up in a normal analytics dashboard.

Access: the fetch that never happens

Every citation starts with a plain HTTP GET, and robots.txt decides whether that GET is welcome. The file format dates back to 1994 and was only standardised in September 2022 as RFC 9309. Three failure modes dominate in practice: a robots.txt that returns a 5xx error (RFC 9309 tells crawlers to treat the whole site as disallowed), a blanket rule written years ago that now blocks retrieval bots along with scrapers, and firewall rules that serve challenge pages to any non-browser user agent.

The fix costs one afternoon: serve robots.txt with a 200 or a 404, never a 500; list the agent classes you accept explicitly; and test the page with a plain GET from outside your network. A blocked fetch is worth 0 citations no matter how good the content is.

Indexability: server HTML versus a JavaScript shell

Retrieval systems work with the HTML the server sends, and only some of them run JavaScript afterwards. Google documents 3 separate phases — crawling, rendering and indexing — in its JavaScript SEO guide, and queues the render pass on its own schedule. Most answer-engine fetchers skip that pass entirely: no render, no hydration, no client router. A single-page app that ships an empty <div id="root"> ships exactly nothing to those systems.

Measure the gap directly: fetch your page with curl, strip the tags and count the sentences that remain. Lekta grades this indexability layer with a JS dependency coefficient between 0 and 1 — the share of visible text that exists only after rendering. Pages under 0.01 are effectively static; pages above 0.5 gamble their entire content on a render pass that most fetchers never run.

Answerability: content shaped for quotation

A fetchable, parseable page still competes on whether its passages answer questions. In a controlled two-document citation test (SIGIR 2026), the passage with concrete data took the first citation by a wide margin: dates, numbers, names and units give a language model something to quote and a fact-checker something to verify. Structured markup does a different job. Google’s AI features guide says AI Overviews and AI Mode need no special schema.org markup, so markup does not win citations on its own; it states facts a parser would otherwise have to guess. An Article object with datePublished and dateModified tells the engine how fresh the page is, and an og:type declaration tells it what kind of document it is reading.

Structured markup is where that shape becomes machine-readable. Declare the document type once and let the vocabulary carry the rest: an Article object states what the page is, an og:type declaration states how it should be previewed, and both describe the document a passage came from.

Recency: a date the machine and the reader both see

In a controlled two-document citation test (SIGIR 2026), the win of a recent date over an old one was the most decisive of the query-independent in-page factors measured — and recency is the signal most often half-implemented. A dateModified stamp in JSON-LD with no date anywhere on the rendered page serves the parser and leaves the reader guessing. Print the date where a reader sees it, keep the markup in sync, and update both when the content actually changes.

Recency does not mean churn. A 2026 vendor test even saw citations rise over 30 days after a date-only change on unchanged text — which is exactly why Lekta reports the age of a page but never scores it: a grade that rewarded fresh dates would reward faking them. What the recency layer grades is whether a date exists at all and whether a reader can see it.

Where to start on a real site

Audit one representative article page, not the homepage. Landing pages are navigation; articles are what answer engines quote. Check the four layers in order — a robots.txt problem invalidates everything downstream, so fixing layers out of order wastes work. The whole loop for one page takes under an hour, and the access and indexability fixes usually carry over to the entire site at once.

Sources

All six references are primary: the RFC text itself, the vocabulary definitions at schema.org and ogp.me, Google’s own documentation on rendering and on its AI features, and the peer-reviewed SIGIR 2026 paper behind the answerability and recency claims. That paper measured which of two candidate documents took the first citation, not whether a real page gets cited at all.

Continue1 of 5 guides
Machine readability: what answer engines actually need from your page — Lekta