What is Web Bot Auth?
2026-05-08
Web Bot Auth is a draft standard that lets a bot sign its requests using HTTP Message Signatures (RFC 9421). The bot's operator publishes a key directory; the site verifies the signature against it. Identity stops being an inference from network position and becomes mathematics.
The early adopters are the agentic fetchers — operator-driven browsers and assistants whose traffic comes from cloud IPs that change too often for published ranges to track. For them, a signature header is the only stable identity they can offer.
Bulk crawlers have been slower to sign, because IP lists already work for them. The likely end state is layered: signatures where they exist, IP verification where they do not, and honest uncertainty everywhere else.