
Cloudflare blocks AI crawlers by default and offers Pay Per Crawl payment model
Cloudflare blocks AI crawlers by default and introduces Pay Per Crawl, letting content creators charge AI services for access. Using HTTP 402 responses and authentication, publishers can allow, charge, or block crawlers with flat prices. This empowers creators to monetize content at scale and supports future dynamic pricing and agent-based access. Pay Per Crawl is in private beta.