# robots.txt for http://www.mi-data.org/ # exclude specific areas from all search engine crawlers User-agent: * Disallow: /2004 Disallow: /2005 Disallow: /2006 Disallow: /2007 Disallow: /2008 Disallow: /admin Disallow: /images Disallow: /inc Disallow: /reports Disallow: /tmp