WeblexBot Our little spider trap caught another scraper who ignores robot.txt and it apparently igonres nofollow...


Who is GSLFBOT? This bot, GSLFbot, does not respect our robots.txt directive and has been banned. IP's... Whois

Lately we have being seeing a lot of spidering traffic from this IP which apparently originates in...
YPBot/Raven1.1.3 bot

YPBot/Raven1.1.3 bot

Just had an interesting bot that got caught in our spiderweb. If you crawl a directory prohibited in the robots.txt you...
Page 1 of 3