User-agent: grub-client Disallow: / User-agent: psbot Disallow: / User-agent: roadrunner Disallow: / User-agent: TurnitinBot Disallow: / #Will disallow all urls on your site User-agent: * Disallow: /cgi-bin/ Disallow: /guestbook.html Disallow: /guestlog.html Disallow: /addguest.html Disallow: /emailme.html Disallow: /Fiction/Rants/FuckSuckingAsshole.html Disallow: /Fiction/Conscience/txt/conscience.txt Disallow: /Fiction/Conscience/html/conscience.html