ScraperWebz.io
Last verified: 2026-07-16 · maintained by Unsourced
The omgili crawler feeds Webz.io's commercial web-data feeds, which are resold including to AI trainers. It provides no citation or referral benefit to crawled sites.
The omgili crawler feeds Webz.io, which packages web content into commercial data feeds resold to many buyers — AI trainers among them. So unlike a single-model trainer, omgili is an intermediary: your page can end up in datasets you never see, with no citation or traffic in return. Sites that don't want their content repackaged and resold typically block it outright.
Webz.io publishes neither an IP-range feed nor a reverse-DNS footprint for Omgili (Webz.io), so there is no way to confirm it by network identity. Treat every request carrying the Omgili (Webz.io) user-agent as an unverified claim, and judge it on what it does rather than the name it gives.
Recommended: block. The omgili crawler feeds Webz.io's resold data feeds — AI trainers among the buyers — so your content can end up in datasets you never see, with no citation or traffic back; sites that don't want it repackaged block it outright.
If you do choose to act in robots.txt (which crawlers honour but don't enforce):
User-agent: Omgili (Webz.io) Disallow: /
Where does my content go if omgili crawls it?
Into Webz.io's commercial data feeds, which are resold to many buyers — AI trainers among them. So your page can end up in datasets you never see, with no citation or traffic back.
Why do sites block the omgili crawler?
Because it's an intermediary that repackages and resells content with no benefit to the source — sites that don't want their material redistributed that way typically block it outright.
Can I verify omgili by IP?
No — Webz.io publishes no IP-range feed and no reverse-DNS host for omgili, so the user-agent can't be confirmed at the network level.
Unsourced audits the requests reaching your site, separates genuine Webz.io traffic from spoofed headers, and tracks whether AI answers cite you.
Check your site free →10-day free trial · no card required · cancel anytime