User-agent: * Disallow: /development # Hide huge zip files Disallow: /doc # Hide huge zip files Crawl-delay: 10