Good to Know: Search Engine Robots, and others, 2017


There's a comprehensive assessment, from a webmaster perspective, of search engine robots detailed in a thread on WebmasterWorld. It also includes other bots that may be legitimate, or acceptable to some.


It includes Bot Names, IPs, UAs and comments about the source and bot behaviour.