Most analytics tools make one quiet assumption that breaks everything: they trust the User-Agent. When your logs show a hit from GPTBot or ClaudeBot, that name comes from a header the visitor sends about itself. It is a claim, not a credential.
Any scraper can put GPTBot in its User-Agent in one line of code, and many do, precisely because publishers wave through anything wearing a familiar badge. So when you write a robots.txt rule or a Cloudflare filter based on those names, you may be doing the opposite of what you think: politely standing aside while an anonymous harvester copies your content under a borrowed identity, all while you believe you are tracking the real AI companies.
The fix is not to trust names harder. It is to stop trusting them at all, and verify identity at the network layer instead.
Unsourced ignores the User-Agent as proof and runs two independent checks. Either one is enough; they are belt-and-braces.
The major AI operators (OpenAI, Anthropic, Google and others) publish the IP ranges their crawlers operate from. If a request claiming to be GPTBot comes from inside OpenAI's published range, that is a strong, hard-to-fake confirmation.
This is the check that catches spoofers hiding on rented servers. It works in two moves. Take the visitor's IP and look up its PTR (reverse-DNS) record to get a hostname. Then resolve that hostname forward, and only trust it if one of the forward addresses is the original IP and the hostname belongs to the operator's own domain, for example openai.com.
Here is why that is hard to fake: a PTR record is set by whoever controls the IP, so an attacker on a VPS can make their address claim to reverse-resolve to crawl.openai.com. But they cannot make openai.com's real forward DNS point back at their server. The forged record fails the round trip. It is the same method Google and Bing have long recommended for verifying their own crawlers. It is a consistency check, not cryptography, and it does not need the hype to work. You can run it on a single bot yourself with the free AI crawler verifier.
Run those checks against your traffic and every AI visitor falls into one of three groups:
One deliberate boundary. Unsourced is an observation layer, not a firewall. We do not block anything on your behalf, and we do not sit in your traffic path making unilateral decisions. What we give you is the proof: which IPs are verified, how often impostors are hitting you, who actually owns the addresses behind the borrowed badges. With that, you update your own stack knowing exactly which crawlers earn their access and which are just wearing the uniform.
The mirage clears the moment you stop reading names and start checking origins. Stop managing your AI footprint on trust, and start managing it on verified identity. Once you can see who is really at the door, the next question is what to do with each one: bait the search bots, keep the trainers, block the rest.
Yes, in one line of code, and many do, precisely because publishers wave through anything wearing a familiar badge. So a robots.txt rule or Cloudflare filter built on those names can do the opposite of what you intend: stand politely aside while an anonymous harvester copies your content under a borrowed identity.
Take the visitor’s IP, look up its reverse-DNS record to get a hostname, then resolve that hostname forward again. Only trust it if a forward address matches the original IP and the hostname belongs to the operator’s own domain. An attacker on a VPS can set their reverse record to claim crawl.openai.com, but cannot make openai.com’s real forward DNS point back at their server. The forged record fails the round trip.
Verified (the network evidence backs the claim), Impostor (the evidence contradicts it — the IP is outside the operator’s range or resolves to someone else), and Unverifiable (not enough to decide — no published range and no usable reverse DNS). We hold the unverifiable ones for review rather than guess, because absence of proof is not proof of a lie.
No. It is an observation layer, not a firewall. It shows you which IPs are verified, how often impostors hit you, and who really owns the addresses behind the borrowed badges. You update your own stack, Cloudflare, nginx or robots.txt, knowing exactly which crawlers earn their access.
Paste a bot name or IP and check it on the spot.
Read it →What to do once you know who is really at the door.
Read it →9,543 verified visits. 5.4% were impostors.
Read it →Unsourced captures which AI assistants cite you, proves which crawlers really fetched your pages, and re-checks after you act — evidence, not a score.
© Unsourced — the evidence layer for AI search.