#---------------------------------------------------------- # AMAN_RedSYS(R) - Fileinfo: # $Author: thomas $ # $HeadURL: file:///var/svn/redsys/tags/v5_0_25/content/robots.txt.in $ # $Revision: 3755 $ # $Date: 2008-04-28 12:43:35 +0200 (Mon, 28 Apr 2008) $ # # Copyright (c) AMAN Media GmbH 1997 - 2008 # http://www.aman.de | http://www.redsys.de #---------------------------------------------------------- # # The use of robots or other automated means to access this RedSYS-generated # site without the express permission of the Siteowner is strictly prohibited. # Notwithstanding the foregoing, the Siteowner may permit automated access to # access certain RedSYS-generated pages but soley for the limited purpose of # including content in publicly available search engines. Any other # use of robots or failure to obey the robots exclusion standards set # forth at is strictly # prohibited. # User-agent: * Disallow: /cache/ Disallow: /imagecache/ Disallow: /admin/ Disallow: /images/ Sitemap: /google_sitemap.xml.gz # If you think, a robot is retrieving pages to agressive and your server does # not get along with this retrieval, please uncomment this for the specified crawler # This example instructs MSNBot to wait 20 seconds before retrieving another # page from this host. If you think that MSNBot is being a bit aggresive this # is a way to have it slow down on your host while still making sure that # the RedSYS pages are indexed. #User-Agent: msnbot #Crawl-delay: 20