Time has started serving approved AI crawlers a machine-readable version of its pages, complete with sponsored answers that human readers never see. The ad experiment will get the headlines. The shift underneath it matters more: the web is being forked by crawler identity, which makes knowing which bot fetched which page something you now have to verify.
Time now serves two versions of the same page. Open a Time URL as a person and you get the normal HTML. Visit the same URL as an approved AI crawler and you get a clean Markdown version built for machines to read. Since June 2026, Time has been converting its pages to Markdown and routing approved bots to that version, and some of those machine pages carry sponsored content: branded FAQ answers that name a sponsor, with an Ally Bank campaign among the first. The AI assistants' crawlers, including ClaudeBot, OAI-SearchBot and PerplexityBot, receive that version. Google's ordinary search crawler receives the same HTML a person sees. A developer, Vincent Schmalbach, spotted the split, and it was reported by The Register and Digiday.
Set the ads aside for a moment. The bigger thing is that a major publisher is now forking its content by crawler identity. What an AI is shown is deliberately different from what a person is shown, and the fork is decided by which bot is knocking. Content is no longer one thing served to everyone. It is routed, per visitor, by identity.
The moment that becomes normal, two questions stop being academic. Which bot actually fetched my page? And which version did it get? If you cannot answer those on your own side, you are trusting a request header, and a header is trivial to fake.
The chain is the same one we keep coming back to, because it is the one that holds up:
That is what Unsourced does: we prove which crawlers really reached you and which assistants cite you, and we keep the two linked. For the current picture of who is really crawling real sites, verified against published ranges and reverse DNS, see our 2026 AI Crawler Report.
The web is quietly splitting into what people see and what machines are fed. Time just turned that split into a paid product. Whether you follow suit or not, you cannot manage a channel you cannot see. Verifying which crawler fetched which page is how you get your eyes back on it.
Not necessarily. Serving a clean, machine-readable version to approved crawlers is a legitimate and increasingly common way to court AI answers. The point for everyone else is simpler: content is now being forked by crawler identity, so knowing which crawler fetched which version becomes something you have to verify rather than assume.
It identifies the visitor. Time routes approved AI crawlers to a Markdown version and serves everyone else the normal HTML. Identity is usually read from the User-Agent and the crawler IP. Because a User-Agent can be faked, any serious version of this needs the crawler verified against the operator's published IP ranges and forward-confirmed reverse DNS.
Record it on your side. Log which verified crawler fetched which URL and when, rather than trusting the request header. That is the only way to prove what an assistant was actually shown. Our crawler verifier checks a single bot, and Unsourced keeps the running record across engines.
Anyone can send a request that claims to be OAI-SearchBot or ClaudeBot. If you route content by the header alone, a scraper or a competitor can collect the version you meant only for the assistants. Verifying identity against published ranges and reverse DNS is what stops that.
Check whether a bot name or IP is really who it claims.
Read it →Who is really crawling real sites, verified against published ranges.
Read it →Why the fetch and the citation are separate events, worth measuring apart.
Read it →Unsourced captures which AI assistants cite you, proves which crawlers really fetched your pages, and re-checks after you act — evidence, not a score.
© Unsourced, the evidence layer for AI search. Our data and findings are free to quote and cite — please attribute to Unsourced and link to unsourced.app.