Published IP ranges and identity verification for every major AI crawler — grounded in live operator feeds and reverse DNS, maintained by Unsourced.
AI crawlers are not interchangeable, and you can't tell them apart by the name they announce — a User-Agent string is trivially spoofed. This directory verifies each major AI crawler against its operator's published IP ranges and forward-confirmed reverse DNS, then answers three things for every bot: who it really is, how to confirm a specific request is genuine rather than an impostor, and whether allowing or blocking it actually helps you.
The crawlers below are grouped by what they do with your content, because that is what should drive the decision. New to choosing what to allow? Start with how to handle AI crawlers — bait, keep, or block.
Crawlers behind AI answer engines that cite and link their sources. Being read by these is how you earn citations and referral traffic — court them rather than block them.
Crawlers that gather data to train future models. They send no traffic today, but blocking them opts you out of what the next generation of models will know about you.
Bulk extractors and resold-data crawlers with no citation or referral benefit — and the bucket where brand-spoofing impostors belong, whatever name they announce.
Want to know whether those crawlers' models cite you? See does AI cite your site, or how Unsourced compares with other AI visibility tools.
10-day free trial · no card required · cancel anytime