Spotting an impersonator
2026-05-22
An impersonator is a request whose user-agent claims a known crawler but whose network position fails verification — 'Googlebot' from a residential broadband address, 'bingbot' from a scraping farm. The claim and the evidence disagree.
The motive is straightforward: search-engine names are skeleton keys. Sites hesitate to block anything calling itself Googlebot, so scrapers wear the costume hoping nobody checks.
Care is required in the other direction too. Operators publish incomplete range lists, and a missing IPv6 table can make a real crawler look fake. A verifier that wants to do no harm treats checked-and-failed as the only damning evidence, and unverifiable as exactly that.