User-agent: * Disallow: /inc/ # go away User-agent: almaden Disallow: / User-agent: ASPSeek Disallow: / User-agent: baiduspider Disallow: / User-agent: dumbbot Disallow: / User-agent: Generic Disallow: / User-agent: grub-client Disallow: / User-agent: htdig Disallow: / User-agent: MSIECrawler Disallow: / User-agent: nexabot Disallow: / User-agent: NPBot Disallow: / User-agent: OWR_Crawler Disallow: / User-agent: psbot Disallow: / User-agent: rabaz Disallow: / User-agent: RPT-HTTPClient Disallow: / User-agent: ScoutAbout Disallow: / User-agent: semanticdiscovery Disallow: / User-agent: TurnitinBot Disallow: / User-agent: Wget Disallow: /