# File to tell robots where I do not want them browsing. # URL: http://tax.cchgroup.com/robots.txt # # 03/03/2003 kriegsmk - file created. # User-agent: * Disallow: /ey02/ # 03/18/2004 and 03/19/2004 # Per Tickets 61618 and 61898 Disallow: /direct/ Disallow: /ecom/ Disallow: /ftw/ Disallow: /mta/ Disallow: /pta/ Disallow: /sita/ Disallow: /suta/ Disallow: /taxbuddy/ Disallow: /TSA/ Disallow: /getcontact/ Disallow: /dependentrebate/ # 11/17/2008 JA Disallow: /Marcom/ Disallow: /lp/kleinrock/libertytax/ # 01/08/2009 EH Disallow: /NR/ # 01/27/2009 KK # only visit between 06:00 and 08:45 UTC (GMT) # this is documented as an "extended" standard, and may not work #Visit-time: 0600-0845 # 01/27/2009 PHI # Set a small delay between requests to 2 sec per doc # only recognized by some bots. # 02/16/2009 EH # Crawl-delay of 2 removed # 03/04/2009 EH Disallow: /ipnetwork # 03/12/2009 EH Disallow: /primesrc/bin/ # 03/31/2009 EH Disallow: /sourcenet/ Disallow: /sourcenet/bin/ # 05/11/09 EH Disallow: /IntelliConnect/ # 05/20/09 RN Disallow: /Microsites/Paperless/Registered/ # 08/12/09 EH Disallow: /cchloginexpress/ # 08/25/09 JA Disallow: /Micro/Paperless/Registered/ # 08/25/09 JA Disallow: /Micro/TZL/Registered/ # 11/10/09 EH Disallow: /lp/