# robots.txt => http://www.your-tags.de/ Sitemap: http://www.your-tags.de/sitemap-index.xml User-agent: * Disallow: *?sort= Disallow: *&sort= Disallow: /index/ Disallow: /rss/ Disallow: /history/ User-agent: Slurp Crawl-delay: 20 User-agent: twiceler Disallow: / User-agent: webalta Disallow: /