AI crawler directory

Published IP ranges and identity verification for every major AI crawler — grounded in live operator feeds and reverse DNS, maintained by Unsourced.

AI Search

• Applebot-Extended — Apple
• ChatGPT-User — OpenAI
• Google-Extended — Google
• OAI-SearchBot — OpenAI
• Perplexity-User — Perplexity
• PerplexityBot — Perplexity
• YouBot — You.com

AI Trainer

• Amazonbot — Amazon
• CCBot — Common Crawl
• ClaudeBot — Anthropic
• cohere-ai — Cohere
• Google-CloudVertexBot — Google
• GPTBot — OpenAI
• Meta-ExternalAgent — Meta

Scraper

• Bytespider — ByteDance
• Diffbot — Diffbot
• Omgili (Webz.io) — Webz.io
• PetalBot — Huawei