# All robots will spider the domain User-agent: * Disallow: # Disallow directory /manager/ User-agent: * Disallow: /manager/ # Disallow directory /assets/ User-agent: * Disallow: /assets/ # Disallow directory /templates/ User-agent: * Disallow: /templates/ # Disallow directories deleted User-agent: * Disallow: /comics/ Disallow: /private/ Disallow: /gone/ Disallow: /beta/ Disallow: /test/ Disallow: /legal/