Cloudflare launches a marketplace that lets websites charge…

Cloudflare launches a marketplace that lets websites charge AI bots for scraping | TechCrunch

Key takeaways:

Cloudflare has launched an experimental marketplace called Pay per Crawl, allowing websites to charge AI bots for accessing their content.
Site owners can set prices for each crawl, block AI bots entirely, or allow them for free — all from a Cloudflare dashboard.
The goal is to give publishers more control and possibly a new revenue stream in the AI era, where bots often scrape content without sending traffic in return.
Cloudflare reports that AI bots like OpenAI’s crawler take far more content than they refer users back — scraping 1,700 times for every one referral.
Several major publishers like TIME, The Atlantic, and Fortune are supporting Cloudflare’s default setting to block AI bots unless granted access.

‌
‌

Added by

Frontend Developer — UI/UX Enthusiast and building scalable web apps

3 days ago