OpenAI, Google, Microsoft and Anthropic back a shared standard for opting out of AI scraping
OpenAI, Google, Microsoft and Anthropic said they will adopt a new standard that lets publishers opt out of being scraped for AI training. The shared "policy token" signal, proposed by Anthropic standards lead Kevin Kelley within the IETF, will live in a website's robots.txt file, so publishers can apply their preferences across multiple AI companies instead of setting them individually.
