Quote Originally Posted by Turgenev
The other nice thing about the robot.txt file (combined with info from webstats) is if you're getting hit by robots from dubious sites, you can ban those robots from accessing your site.
Bad bots tend to ignore the robots.txt file. The best way to block bad bots is to IP ban them. If however it is a bunch of copies on different IPs you might have to contact your host about setting a limit on how many pages can be accessed by one IP in a given time period (bad bots like to spam page requests really fast).