The bot with user agent Linguee Bot (http://www.linguee.com/bot; bot@linguee.com) just downloaded approximately 2 GiB of stuff from a ViewVC-listing on one of my servers. Just because it’s stupid, but also because it’s so cheeky to ignore robots.txt. So be it. I can indeed match the user agent string … from now on it receives HTTP status code 403.
// Oliver
I was trying to hg clone https://vcs.assarbad.net/ddkbuild/ or https://vcs.assarbad.net/ddkwizard/, but keep getting “abort: HTTP Error 400: Bad Request”. Have no problem accessing them with the web interface, but there is no way to download files there.