Blocking Test Pages Enmasse on Sub-domain

[CeeC-Blogger]

Hello,

We have thousands of test pages on a sub-domain of our site. Unfortunately at some point, these pages were visible to search engines and got indexed.  Subsequently, we made a change to the robots.txt file for the test sub-domain. Gradually, over a period of a few weeks, the impressions and clicks as reported by Google Webmaster Tools fell off for the test. sub-domain.

We are not able to implement the no index tag in the head section of the pages given the limitations of our CMS.

Would blocking off Google bot via the firewall enmasse for all the test pages have any negative consequences for the main domain that houses the real live content for our sites (which we would like to of course remain in the Google index).

Many thanks

[KeriMorgret]

If you want nothing on that test subdomain indexed, verify that subdomain as its own site in Google Webmaster Tools, exclude that subdomain from being indexed in robots.txt, then request removal of that site (subdomain) in GWT.

And consider setting up a page monitor like https://polepositionweb.com/roi/codemonitor/index.php on the robots.txt of your test site (and live site). It'll check the contents of those pages once a day, and email you if there's a change. Handy if there are multiple people working on the site.

[StreamlineMetrics]

I'm a bit confused. Didn't blocking the test subdomain with the robots.txt already accomplish what you are trying to do? Or are the test pages still somehow indexed? Or is your main site affected by the robots.txt? Anyway, I would suggest using the .htaccess file to block search engines from accessing the subdomain rather than a firewall - http://stackoverflow.com/questions/6738896/excluding-testing-subdomain-from-being-crawled-by-search-engines-w-svn-reposit