At the heart of Cloudflare’s concern is Google’s use of artificial intelligence to extract, process, and sometimes even summarize online content. While traditional web crawlers like Googlebot have operated under longstanding industry norms—adhering to standards like robots.txt to respect the wishes of website owners—Cloudflare says the new wave of AI-enabled crawlers doesn’t always follow these same conventions.

“Increasingly, we’re seeing AI agents from major platforms using new mechanisms to grab content—sometimes ignoring or working around existing content-protection signals,” said a Cloudflare spokesperson. “This challenges the balance between innovation and the rights of site creators.”