Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

Blocking subdomains with Robots.txt file

Technical SEO

1001

paulwatley Subscriber last edited by

We noticed that Google is indexing our pre-production site ibweb.prod.interstatebatteries.com in addition to indexing our main site interstatebatteries.com. Can you all help shed some light on the proper way to no-index our pre-prod site without impacting our live site?
1 Reply Last reply
Reply Quote 0
PaulM01 last edited by

Placing a robots.txt in the root of the subdomain will do it:

User-agent: *
Disallow: /
1 Reply Last reply
Reply Quote 1

Got a burning SEO question?

Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.

Start my free trial

Browse Questions

View

From

Sorted by

With category

Explore more categories

Related Questions

How to sunset language subdomains that we don't want to support anymore?

We have a primary domain www.postermywall.com. We have used subdomains for offering the same site in different languages, like es.postermywall.com, fr.postermywall.com etc. There are certain language subdomains that have low traffic and are expensive to get translated. We have decided to sunset 3 subdomains that match that criteria. What is the best way of going about removing those subdomains? Should we just redirect from those subdomains to www.postermywall.com? Would that have any negative impact on our primary domain in Google's eye etc.? Anything other than a redirect that we should be considering?
Technical SEO | | 250mils

0
Blocking subdomains without blocking sites...

So let's say I am working for bloggingplatform.com, and people can create free sites through my tools and those sites show up as myblog.bloggingplatform.com. However that site can also be accessed from myblog.com. Is there a way, separate from editing the myblog.com site code or files, for me to tell google to stop indexing myblog.bloggingplatform.com while still letting them index myblog.com without inserting any code into the page load? This is a simplification of a problem I am running across. Basically, Google is associating subdomains to my domain that it shouldn't even index, and it is adversely affecting my main domain. Other than contacting the offending sub-domain holders (which we do), I am looking for a way to stop Google from indexing those domains at all (they are used for technical purposes, and not for users to find the sites). Thoughts?
Technical SEO | | SL_SEM

1
Content on subdomain...

We recently moved our Wordpress site to a new host (WPEngine). We had forums on the old web host, which we need to migrate to a new forum platform (Xenforo) and integrate into the WP site. Since WPEngine only allows Wordpress on their servers, we need to install the forums at another web host, on one of our other domains. We might point to the forums through a subdomain, like this: forums.our-primary-domain.com The main reason we're re-installing the forums is for SEO value. HOWEVER, since our forum content will be on another domain, will we have an issue? If so, is there a workaround that would give us 'credit' for that content? Thanks much.
Technical SEO | | jmueller0823

0
Robots.txt checker

Google seems to have discontinued their robots.txt checker. Is there another tool that I can use to check my text instead? Thanks!
Technical SEO | | theLotter

0
Where Is This Being Addended to Our Page File Names?

I have worked over the last several months to eliminate duplicate page titles at our site. Below is one situation that I need your advice on. Google Webmaster Tools is reporting several of our pages with
duplicate title such as this one: This is a valid page at our Web store: http://www.audiobooksonline.com/159179126X.html This is an invalid page that Google says is a duplicate of the one above: http://www.audiobooksonline.com/159179126X.html?gdftrk=gdfV2138_a_7c177_a_7c432_a_7c9781591791263 Where might the code ?gdftrk=.... be coming from? How to get rid of it?
Technical SEO | | lbohen

0
Robots txt

We have a development site that we want google and other bots to stay out of but we want roger to have access. Currently our robots.txt looks like this: User-agent: *
Disallow: /cgi-bin/
Disallow: /development/ What would i need to addd or change to let him through? Thank you.
Technical SEO | | LadyApollo

0
Keywords in file names vs folder names

We understand the value of a keyword phrase included in the URL. Is there more value to having that phrase in the folder name of the URL or the file name or does it matter? Example: http://www.biztoolsone.com/website-design.php or http://www.biztoolsone.com/website-design/ Which is best? Thanks, Wick Smith
Technical SEO | | wcksmith

0
Switching subdomains

A few years ago our company decided to merge all its websites (magazine brands) as sub-domains under one new root domain. So the current situation is like this: brand1.rootdomain.com
brand2.rootdomain.com
brand3.rootdomain.com
... For the moment the rootdomain has a domain authority of 66. In a few weeks we would like to switch that rootdomain to the strongest (highest trust, pagerank,...) brand. So we get this: www.brand1.com
brand2.brand1.com
brand3.brand1.com Before we make the switch i'll have to make a pro and con list. So I hope I can get some advice for you guys if this is a good idea or not.
Technical SEO | | WDN

0