Identity is not enough; behavior matters
Cloudflare has added a new layer to its Verified Bot program. The company's analogy is simple: a driver's license tells you who someone is and that they passed a test, not whether they will drive responsibly in the future. The same holds for bots, where signing requests and proving identity alone is not enough.
According to Cloudflare, becoming a Verified Bot broadly requires three things: identifying yourself accurately and securely, doing only what you say you will do, and not engaging in abusive behavior. Bots sign requests using Web Bot Auth, and BotBase catalogues them in an open format.
Risk is momentary, trust accumulates
Cloudflare separates risk from trust: risk is the likelihood that a single request may be harmful and can change moment to moment, while trust is built over time through consistent behavior. The company recalls de-listing Perplexity's bot from its Verified bot list in 2025 after it sought to access sites that had opted out of scraping and then tried to circumvent controls when blocked.
What is being measured?
The new system measures how often a crawler respects site preferences. The initial signals it evaluates are:
- Does the bot respect instructions?
- Did it respond to direct feedback?
The measurement focuses on repeated patterns of behavior rather than isolated violations. robots.txt is the primary way site owners communicate preferences, and the system checks whether observed traffic is consistent with those preferences over time. Cloudflare also stresses that robots.txt sets only minimum rules, and that anything not stated is not implicitly allowed.
Load is a signal too
Bots can stay within allowed paths while still placing substantial load on the sites they visit. One of the clearest ways a site can push back is an HTTP 429 ("Too Many Requests") response. The system also evaluates how a bot responds to such direct signals.
What comes next
Cloudflare says it will translate these observations into an explainable trust band rather than a permanent judgment: a range showing how consistently a bot respects site preferences and which observed behaviors informed that assessment. The company plans to share these scores publicly on Radar and to expand its measurement methods. The goal is to give responsible bot operators a way to demonstrate that trust is being upheld over time.
