Cloudflare just dropped a staggering number that shows how aggressively AI companies are scraping the web. The internet infrastructure giant has blocked 416 billion AI bot requests since July 1, CEO Matthew Prince revealed at WIRED's Big Interview event. The figure highlights the massive scale of AI data harvesting and Cloudflare's role as the web's gatekeeper in the battle between content creators and AI companies.
The numbers are mind-bending. Cloudflare has intercepted 416 billion AI bot requests in just five months, a figure that CEO Matthew Prince shared publicly for the first time at WIRED's Big Interview event Thursday. That's roughly 2.8 billion blocked requests every single day since the company launched its Content Independence Day initiative in July.
The scale reveals just how voraciously AI companies are attempting to hoover up web content. "The business model of the internet has always been to generate content that drives traffic and then sell either things, subscriptions, or ads," Prince told WIRED's Brian Barrett. "What people don't realize is that AI is a platform shift. The business model of the internet is about to change dramatically."
Cloudflare's position as a major CDN protecting millions of websites gives it unprecedented visibility into this digital gold rush. The company's Content Independence Day initiative launched with a simple premise - AI crawlers should be blocked by default unless companies pay for access. Since then, the blocking tools have become increasingly popular with publishers desperate to protect their content.
But it's Google that's drawing the sharpest criticism from Prince. The search giant made a controversial decision to combine its search and AI crawlers into one system, creating an impossible choice for content creators. Block the AI scraper and you disappear from Google search. Allow the scraper and your content trains the next generation of AI models for free.
"You can't opt out of one without opting out of both, which is a real challenge - it's crazy," Prince said. "It shouldn't be that you can use your monopoly position of yesterday to leverage a monopoly position in the market of tomorrow."
The data Cloudflare shared reveals just how dominant Google's web crawling operation has become. The company sees 3.2 times more pages than OpenAI, 4.6 times more than , and 4.8 times more than either or Anthropic. "They have this incredibly privileged access," Prince noted.
