How can I make Google Webmaster Tools see the robots.txt file when I am doing a .htacces redirec?

[RalphinAZ]

We are moving a site to a new domain.  I have setup an .htaccess file and it is working fine.  My problem is that Google Webmaster tools now says it cannot access the robots.txt file on the old site.  How can I make it still see the robots.txt file when the .htaccess is doing a full site redirect?

.htaccess currently has:

Options +FollowSymLinks -MultiViews
RewriteEngine on
RewriteCond %{HTTP_HOST} ^(www.)?michaelswilderhr.com$ [NC]
RewriteRule ^ http://www.s2esolutions.com/ [R=301,L]

Google webmaster tools is reporting:

Over the last 24 hours, Googlebot encountered 1 errors while attempting to access your robots.txt. To ensure that we didn't crawl any pages listed in that file, we postponed our crawl. Your site's overall robots.txt error rate is 100.0%.

[benjaminspak]

Possible Solitions for your problem:

.htaccess authentication blocking robots.txt

http://www.irishwebmasterforum.com/server-technical-administration-tips-and-queries/10892-htaccess-authentication-blocking-robots-txt.html

301 redirect. How to make an exception for the robots.txt

http://forum.cs-cart.com/topic/23747-301-redirect-how-to-make-an-exception-for-the-robotstxt/

1. Canonical robots.txt

http://digwp.com/2011/03/htaccess-wordpress-seo-security/

General .htaccess tutorials: http://httpd.apache.org/docs/2.0/howto/htaccess.htmlhttp://httpd.apache.org/docs/2.0/misc/rewriteguide.html

[RalphinAZ]

Thank you that seems to be working.

[matbennett]

You could add an exception to the htaccess to allow the robots to be loaded.  You would do this with by adding another condition.  I'd use something like:

<code>Options +FollowSymLinks -MultiViews
RewriteEngine on
RewriteCond %{REQUEST_URI} !/robots.txt
RewriteCond %{HTTP_HOST} ^(www\.)?michaelswilderhr\.com$ [NC]
RewriteRule ^ [http://www.s2esolutions.com/](http://www.s2esolutions.com/) [R=301,L]</code>

Disclaimer: I am lucky enough to have people at work who check these things. This hasn't been checked! Use at your own discretion

However I'll admit that I've never used this. I just stick the 301 in and it all seems to work out fine.  Probably done it on hundreds of domains over the years.