# Robots.txt file created by http://www.webtoolcentral.com # For domain: http://www.thefollowon.com # All robots will spider the domain User-agent: * Disallow: User-agent: Mediapartners-Google* Disallow: # Disallow directory /cgi-bin User-agent: * Disallow: /cgi-bin # Disallow directory /cgi_data User-agent: * Disallow: /cgi_data