# robots.txt for http://www.christinaflordh.se/ # Comments to the webmaster should be posted at # Allows only major search engines and known friendly spiders # Major Search Engines and Known Friendly Spiders (allowed) User-agent: Googlebot Disallow: User-agent: MSNBot Disallow: User-agent: Nutch Disallow: User-agent: Openbot Disallow: User-agent: Openfind Disallow: User-agent: ia_archiver Disallow: User-agent: Slurp Disallow: User-agent: Teoma Disallow: User-agent: Gigabot Disallow: User-agent: Scrubby Disallow: User-agent: Robozilla Disallow: User-agent: Mozilla/3.0 (compatible;miner;mailto:miner@miner.com.br) Disallow: User-agent: WebFerret Disallow: User-agent: Gulliver/1.1 Disallow: User-agent: JCrawler/0.2 Disallow: User-agent: Informant Disallow: User-agent: InfoSeek Robot 1.0 Disallow: User-agent: Infoseek Sidewinder Disallow: User-agent: InfoSpiders/0.1 Disallow: User-agent: Lycos/x.x Disallow: User-agent: Lycos_Spider_(T-Rex) Disallow: User-agent: Scooter/1.0 Disallow: User-agent: WWWWanderer v3.0 Disallow: User-agent: WebCopy/ Disallow: User-agent: WebCrawler/3.0 Robot libwww/5.0a Disallow: User-agent: WebFetcher/0.8, Disallow: User-agent: weblayers/0.0 Disallow: # Everyone Else (NOT allowed) User-agent: * Disallow: /cgi-bin/ Disallow: /bilder/ Disallow: /login/ Disallow: /error.htm Disallow: /stats/awstats Disallow: /scripts/