# Robots.txt # All robots will spider this domain User-agent: * Disallow: /error/ Disallow: /generator/ Disallow: /stats/