# # This is the robots.txt file. It should be in the root (/) # of your web site. # # This file is used to tell search engine crawlers what # directories they should ignore when indexing. # # Lines beginning with # are comments, and are not processed # by crawlers. # # The format is a series of records. Each record begins with # a User-agent: statement, and is followed by at least one # Disallow: statement. The preset records below demonstrate # this format. # # Records are separated by blank lines # # For further information, visit # http://info.webcrawler.com/mak/projects/robots/norobots.html # # Edit this record as appropriate for your site. Add any # other directories you want crawlers to ignore. User-agent: * Disallow: /AtWork/ User-agent: * Disallow: /MyCommunity/ User-agent: shs-crawler Disallow: