Thursday, May 31, 2007

TMCrawler

Host: 128.241.20.206
Agent: TMCrawler

This bad-bot visited a site, grabbed the robots.txt twice and started to follow links at a very leisurely pace: just sub-directories over a couple of hours.

Then it started to explore a directory in a systematic fashion. It picked a sub-directory and began a numerical search: /directory/0, then /directory/1, and so on. I caught this bad-bot quickly because I have any 404 errors emailed to me immediately. I suspect it would have tripped a trap soon enough though.

Others have reported this bot as a WTF is this thing doing?

Based on its activites, I have simply decided to deny it access to my sites.

No comments: