AI Pages

:::summarybox learn What an AI page actually is, and why the concept exists at all The infrastructure parallel: robots.txt and sitemap.xml, but for AI consumption How Trakkr generates AI pages and serves them only to AI crawlers What gets transformed: schema, key facts, FAQ structure, entity tags How to install the middleware on Cloudflare, Vercel, WordPress, and seven other platforms How to measure whether it's actually moving the needle on visibility

note

Most pages on the web are built for one audience: a human with a browser. The styling, the navigation, the JavaScript that fetches content after the page loads, the marketing copy that hooks before it informs, all of it is shaped around what works for a person reading on a screen.

An AI page is a version of that same page built for a different audience: a language model. No styling, no navigation, no JavaScript. Just the underlying facts of the page rendered as clean, structured, machine-legible HTML, with explicit signals about what type of content this is, what the key data points are, and which entities it references.

The reason this needs to exist is that the audience for the web has quietly split in two. When ChatGPT or Claude or Perplexity crawls a page, they're not loading it in a browser. They don't run your JavaScript, they don't wait for content to render, and they don't have time to figure out which div in your hero section actually contains the headline. They get whatever HTML your server returns in the first round-trip, and if that HTML is empty (as it is for most JavaScript-rendered sites), they walk away with nothing.

The parallel that helps people get this is robots.txt and sitemap.xml. Nobody types those URLs into a browser. They exist exclusively for crawlers, written in a format crawlers can parse, served alongside your normal site. AI Pages is the same idea applied one level up: instead of a sidecar file that tells crawlers where to go, it's a sidecar version of each page that tells them what's there.

What an AI page actually contains

An AI page is your existing page, transformed. Not different content, the same content in a different shape.

The transformation pipeline takes the rendered HTML of your page (after JavaScript has run, on Trakkr's side, not the crawler's), strips out the parts that don't matter to a language model, and adds explicit structure to the parts that do. The result is a smaller, denser, and more legible version of the same information.

Here's the kind of thing that gets stripped: analytics scripts, tracking pixels, animation libraries, CSS bundles, decorative wrappers, lazy-load placeholders, A/B test flags, the whole rendering scaffolding that exists to make a page look good and load fast for humans. None of it helps a model understand what the page is about.

Here's what gets added or made explicit: schema.org markup that labels the page as a Product or Article or FAQ, key facts pulled out and tagged (<span itemtype="price">$130</span>), an FAQ block that reformats your content into the Q&A shape AI loves to cite from, an AI-readable summary that distills the page in plain language, and entity tags around every brand, product, person, and place you mention.

The five enhancement features are documented on their own page. The short version is that each one targets a specific weakness in how raw HTML communicates with a language model.

How Trakkr serves AI pages to crawlers (and only to crawlers)

This is the part that sounds like cloaking but isn't. The mechanism is a small piece of middleware you deploy on your hosting platform. It runs in front of every request, checks the user agent, and routes accordingly:

For a human visitor, the middleware adds under 10ms of latency and then steps out of the way. Your real site loads exactly as it always did. For an AI crawler, the middleware makes a request to Trakkr's optimization service, gets back the cached AI page, and serves that as the response.

The cache is the thing that makes this fast. The first time GPTBot visits yoursite.com/products/widget, Trakkr does the full transformation (render the page, strip the noise, add the structure, save the result) which takes a couple of seconds. Every subsequent visit to that URL by any AI crawler gets the cached version in around 100ms. The cache lives for seven days, so a popular page might get re-optimized once a week.

Trakkr generates code snippets for nine integration paths: Cloudflare Workers, Vercel Edge Middleware, Netlify Edge Functions, Next.js middleware, AWS Lambda@Edge, a WordPress mu-plugin, Node.js / Express, Nginx with OpenResty, and a Cloudflare DNS proxy approach for platforms (Shopify, Squarespace, Wix, Webflow) that don't support server-side middleware natively. The setup wizard on the AI Pages page generates the snippet for your stack with your API key already baked in.

Isn't this just SEO?

SEO is the practice of making your site rank well in search results: title tags, internal linking, page speed, backlinks, keyword targeting. The audience is Google's ranking algorithm, and the goal is to be one of the ten blue links it shows next to a query.

AI Pages is the practice of making your content readable, citable, and trustworthy to a language model that's generating an answer. The audience is GPTBot, ClaudeBot, PerplexityBot. The goal is to be in the answer the model writes, or in the citation list underneath it.

Search engines and AI crawlers behave differently, and AI Pages never serves transformed content to Googlebot or Bingbot. Your search rankings are untouched. The transformation only fires when the user agent matches a known AI crawler. If you removed AI Pages tomorrow, your SEO would not notice.

What you'll see in the dashboard

The AI Pages page in Trakkr is an operational dashboard, not a configuration screen. Once setup is done, it's mostly for watching what AI crawlers are doing on your site.

Three tabs:

Tab	What it shows
Overview	Live crawler activity, requests over time, crawler breakdown by company, response time chart, top pages, recent crawl log.
Pages	Per-URL view of which pages have been optimized and cached, sortable by visits or last-crawled time.
Usage	Monthly request count vs your plan limit, overage settings, spending cap, billing detail.

The thing to watch most often is the crawler breakdown on the Overview tab. It tells you which AI models are actually showing up on your site. If GPTBot visits 200 times a week and ClaudeBot visits twice, the OpenAI side of your visibility story has more recent data to draw on than the Anthropic side.

The recent crawl log is useful for a different reason: it shows whether AI Pages is doing its job. Each row has a status (Cache Served, Cache Created, or Error) and a response time. A healthy site is mostly Cache Served at sub-150ms with no errors.

What you'll see in your AI visibility, eventually

This part requires patience. Crawling is not the same as ingesting, and ingesting is not the same as recommending. There's a real lag between "Trakkr served an AI page to GPTBot" and "ChatGPT started mentioning your brand."

Model family	Typical lag from crawl to visibility
Perplexity, ChatGPT Search	Hours to a few days (live search)
ChatGPT main model	Days to weeks (incremental updates)
Claude	Weeks to months (training cycles)
Gemini, Copilot	Days to weeks

The right way to evaluate AI Pages isn't "did my score change this week?" It's "over the next quarter, on the prompts where I was invisible, am I starting to appear?" Pair AI Pages with the Prompts page so you have a baseline to measure against. If you started AI Pages on March 1 with a 32% visibility score on a key prompt and you're at 47% in May, that's signal. If a competitor that hasn't done this work is flat over the same window, that's stronger signal.

What it costs

AI Pages bills on AI crawler requests that received an optimized response. Human traffic doesn't count. Static assets (.css, .js, images) don't count. Errors don't count. Cache hits and cache misses both count once.

Plan	Included requests	Overage
Growth	2,500 / month	$5 per 1,000
Growth + Prism add-on	10,000 / month	$5 per 1,000
Scale	10,000 / month	$5 per 1,000
Enterprise	Custom	Custom

Overage billing is off by default. When you hit your limit, AI Pages stops serving optimized content and AI crawlers get your normal site until the month resets. If you'd rather keep serving and pay for the extra requests, turn overage on in the Usage tab and (optionally) set a spending cap so you don't get surprised.

A small content site usually fits well inside Growth. A large e-commerce catalog (5,000+ URLs) or anything that crawlers visit aggressively (news, frequently-updated docs) usually wants Scale or the Prism add-on. The Usage tab shows your trajectory inside the month, so you can see early whether you're going to run out.

Common questions

Is this safe? Is it cloaking?

No. Cloaking, as Google defines it, is showing search engines content that differs from what humans see, with the intent of manipulating rankings. AI Pages doesn't do either thing. Googlebot, Bingbot, and every other SEO crawler always receive your original site, untouched. The version served to AI crawlers isn't different information either, it's the same information in a more machine-legible format. The closest real-world analogy is a print stylesheet or AMP version of a page: same facts, different rendering for a different audience.

:::details Want the full reasoning? Here's the precise breakdown of why this doesn't qualify as cloaking. Google's actual cloaking definition (Search Central Spam Policies) has three elements that all need to be present:

Different content is served.
The different content goes to a search engine.
The intent is to manipulate search rankings or mislead users.

AI Pages clearly fails elements 2 and 3, and arguably fails element 1 too. Going through each:

1. "Different content." What an AI page contains is your existing page's facts re-expressed as structured HTML: schema.org markup, an extracted key-facts list, an FAQ block built from your own content, an entity-tagged summary. Nothing is removed, replaced, or contradicted. The technical term is additive, not substitutive. If you diff the human page and the AI page, the AI page is a strict superset of the same underlying claims. You can verify this yourself with the Test Prism tool on the Cache tab: paste a URL and compare the two HTML outputs side by side.

2. "Served to a search engine." AI Pages routes by user agent. Googlebot, Bingbot, DuckDuckBot, Yandex, Baidu, and every other traditional search crawler get the original page. The middleware only intercepts known AI assistant crawlers (GPTBot, ClaudeBot, PerplexityBot, etc.), and you control that list in the Crawlers tab. Your search rankings are mechanically untouchable from this layer.

3. "Intent to manipulate or mislead." The transformation makes the same facts easier for a language model to parse and cite accurately. The goal is fewer misrepresentations of your brand in AI answers, not more. The audit trail is explicit: the injected block is wrapped in <div data-prism-content="true"> so any auditor (or AI lab) can identify the optimization on inspection.

The industry parallel. Edge-side content adaptation has been mainstream SEO practice for years. Major edge platforms support serving differentiated responses by request signal, and schema injection at the edge is documented as standard technical SEO (Search Engine Land). Serving AI crawlers a cleaner version of the same content is the dominant pattern in the 2026 generative-search optimization category.

Two honest caveats. AI Pages is conservative by design, but two settings are yours to choose:

Google-Extended. Google-Extended is Google's AI training crawler, distinct from Googlebot (search). It's enabled by default in AI Pages because most customers want to be ingested by Google's AI products. If you want zero ambiguity about Google policy ever applying, you can uncheck Google-Extended in the Crawlers tab. Doing so means Google's AI training pipeline sees your original HTML instead of the optimized version.
Content accuracy. The optimization features are grounded in your page's existing content, with length checks and type guards to reject hallucinated output. We recommend spot-checking the AI page on a few key URLs after enabling. Partly to confirm the FAQ and entity extraction read accurately, partly because that's the single thing that would shift this from "optimization" to actual misrepresentation.

The clean test. If a reasonable auditor compared the AI page and the human page for any URL on your site, would the facts agree? With AI Pages, the answer is yes. That's the whole design constraint, and it's what separates legitimate AI optimization from the "AI-targeted cloaking" attacks (The Hacker News, Oct 2025) where attackers feed crawlers fabricated facts the real site doesn't support.

note

What happens to my site if AI Pages goes down?

The middleware fails open. If Trakkr's API is unreachable, slow to respond, or returns an error, the middleware drops back to serving your original page. AI crawlers get what they would have gotten without AI Pages installed. Human visitors are never affected because they don't go through the optimization path in the first place.

Can I exclude specific pages?

Yes. In the AI Pages settings, you can configure path-level exclusions (for example, /admin/* or /checkout/*) that skip optimization entirely. By default the middleware already skips static assets, API routes, and a list of common non-content paths.

Will it slow my site down?

For humans, no, the user-agent check adds well under 10ms and your real page serves normally. For AI crawlers, a cache hit returns in about 100ms and a cache miss in about 2 seconds, but cache misses only happen on the first visit to each page or after the seven-day cache expires.

Does this affect what humans see?

No. The middleware checks the User-Agent header before doing anything. Any request that isn't from a known AI crawler is passed through to your origin unchanged. There is no scenario where a human visitor receives the AI page version.

What if my page changes? Will the cached AI page get stale?

Caches expire after seven days, so the next AI crawler visit after expiry triggers re-optimization. If you ship a meaningful content change and want it picked up sooner, you can regenerate the API key (or, soon, manually invalidate specific URLs from the Pages tab) to force a refresh.

Does it work with Shopify, Squarespace, Webflow, or Wix?

Not natively (those platforms don't let you run server-side middleware), but yes via the Cloudflare DNS proxy path documented in Installation. You put Cloudflare's free tier in front of your existing site, deploy the Cloudflare Worker, and the rest works the same.

What's the difference between AI Pages and the Optimize page?

Optimize gives you a list of changes to make to your real site, things you'd hand to a developer or do yourself in your CMS. AI Pages serves an optimized version of your site to AI crawlers without changing the real one. They're complementary: Optimize improves the page for all audiences (humans and AI), AI Pages adds a dedicated layer for the AI audience specifically.