weblexbot

WeblexBot

WeblexBot Our little spider trap caught another scraper who ignores robot.txt and it apparently igonres nofollow...