User-agent: Amazonbot Crawl-delay: 10 User-agent: GPTbot Crawl-delay: 10 User-agent: * Allow: / Disallow: /date/2010*$ Disallow: /date/200*$ Disallow: /date/19*$