# /robots.txt file for http://www.boystown.com.au # Hash preceeds a comment # To exclude certain pages specifically add this meta tag into the head # # All robots will spider the domain #User-agent: * #Disallow: # To disallow robots access to certain directories add # the directories on a separate line starting with # Disallow: # Disallow directory(s) User-agent: * Disallow: /BTIntranet/ Disallow: /cyphone/ Disallow: /downloads/vac/ Disallow: /email/ Disallow: /emailcounselling/ Disallow: /errorpages/ Disallow: /images/email/ Disallow: /includes/ Disallow: /intranet/ Disallow: /Japanese/ Disallow: /JavaAplet/ Disallow: /JavaMenus/ Disallow: /khl/ Disallow: /lifestyle/ Disallow: /lifestyle/multichance/ Disallow: /lockup/ Disallow: /macros/ Disallow: /macrosResources/ Disallow: /mail/ Disallow: /pcaudit/ Disallow: /prestigecar/ Disallow: /retailmenu/ Disallow: /script/ Disallow: /security/ Disallow: /styles/ Disallow: /teamworkplace/ Disallow: /trim/ Disallow: /vacancies/ Disallow: /virus/ Disallow: /webdesign/ Disallow: /wuimages/