Tech Trends 📰

Cloudflare launches tool to help website owners monetize AI bot crawler access

Cloudflare has launched a new tool that allows website owners to block AI crawlers or charge them using a “pay per crawl” model.. This move aims to help content creators and publishers monetize access to their work as AI bots increasingly collect data without driving traffic back to original sources.. The initiative is backed by major publishers and platforms, including Condé Nast, Associated Press, Reddit, and Pinterest.. Cloudflare’s data shows Google’s crawl-to-referral ratio has worsened (18:1), while OpenAI’s is drastically worse (1,500:1), indicating AI bots are taking content without returning user traffic.. The tool is part of a broader effort to protect online content creators as traditional ad revenue from search traffic declines due to AI-generated summaries and chatbot responses..

Cloudflare launches a marketplace that lets websites charge AI bots for scraping | TechCrunch

Cloudflare has launched an experimental marketplace called Pay per Crawl, allowing websites to charge AI bots for accessing their content.. Site owners can set prices for each crawl, block AI bots entirely, or allow them for free — all from a Cloudflare dashboard.. The goal is to give publishers more control and possibly a new revenue stream in the AI era, where bots often scrape content without sending traffic in return.. Cloudflare reports that AI bots like OpenAI’s crawler take far more content than they refer users back — scraping 1,700 times for every one referral.. Several major publishers like TIME, The Atlantic, and Fortune are supporting Cloudflare’s default setting to block AI bots unless granted access..

Cloudflare Reveals AI Labyrinth to Counter Automated AI Attacks

Cloudflare introduced AI Labyrinth to fight unauthorized AI bots that scrape website data.. It works by generating AI-created web pages to mislead and trap bots, wasting their resources.. The system is easy to activate from the Cloudflare dashboard and works automatically.. It uses Cloudflare’s Workers AI and R2 storage to create and store factual but irrelevant content.. Hidden links on websites guide bots into the labyrinth, helping identify and block them effectively..