Host: 63.100.163.70
This bot, disguised as MSIE tried to rip through one of my sites, and ran right into a bot trap. It started off looking like a regular browser: it loaded the site's root index and the page's .css. It didn't load the robots.txt.
Nevertheless, several things in combination gave it away as a disrespectful bot or ripper:
- It wasn't loading the page's associated binaries, i.e. images and so on
- It wasn't loading javascript (everyone knows that MSIE can't do much without it)
- It was crawling pages at three pages per second
- On closer inspection, it only loaded one of the two .css files on the index page
- It tried to follow links that were commented out in the page's mark-up
- It ran into a bot trap that a normal user wouldn't see.
Whois says 63.100.163.70 belongs to:
UUNET Technologies, Inc.
22001 Loudoun County Parkway
Ashburn, VA, 20147, US
No comments:
Post a Comment