Matt/Ryan-

Great discussion, thanks for the input. The staging.domain.com is just one of the domains we don't want indexed. Some of them still need to be accessed by the public, some like staging could be restricted to specific IPs.

I realize after your discussion I probably should have used a different example of a sub-domain. On the other hand it might not have sparked the discussion so maybe it was a good example

Interesting, hadn't thought of using htaccess to block Googlebot.Thanks for the suggestion.

Thanks Ryan. So you don't see any issues with de-indexing the main site if I created a second robots.txt file, e.g.

http://staging.domin.com/robots.txt

User-agent: *
Disallow: /

That was my initial thought but when Google announced they consider sub-domains part of the TLD I was afraid it might affect the htp://www.domain.com versions of the pages. So you're saying the subdomain is basically treated like a folder you block on the primary domain?

Now that Google considers subdomains as part of the TLD I'm a little leery of testing robots.txt with something like:

staging.domain.com
User-agent: *
Disallow: /

in fear it might get the www.domain.com blocked as well. Has anyone had any success using robots.txt to block sub-domains? I know I could add a meta robots tag to the staging.domain.com pages but that would require a lot more work.