booter – bots & crawlers manager – wordpress plugin

The plugin uses number of existing technologies which are known by crawlers and bots and takes them one step forward and almost completely automatically..   To allow the plugin to function correctly, you must follow the instructions manually enter some data ..   Booter allows you to limit the amount of requests from crawlers and bots, if or when they exceed the specified amount of requests per minute, it will be rejected for period of time..   Rejecting links that we do not want in the fastest way, not by just blocking but by sending HTTP status code to make search engines forget them...   Watch the 404 log, try to find common parts in URLs that repeats most often..   Enter the common parts to the reject links page, and ensure the rejection code is 410..   Activate the plugin through 'Plugins WordPress.   The plugin will start rate limiting as soon as it's activated, however it's recommended to update the settings to suit your needs, under 'Settings -> 'Booter Crawlers Manager menu...   Interested in development.., check out SVN repository, or subscribe to the development log by RSS...   Added option to add rejected links to robots txt.   Read more