User-Agent: * Crawl-delay: 2 Request-rate: 1/2 Disallow: