Welcome to the Q&A Forum
Browse the forum for helpful insights and fresh discussions about all things SEO.
Hi Paul,
Some great advice.
In the turn of the New Year when we have more resources we are going to take action following your plan - it is a lot more work but makes perfect sense.
Again, thank you for pointing out that I need to remove the robots.txt block so that the page commands can be discovered - I will get onto this straight away.
Thank you once again.
Lloyd
Great, thank you for your advice Jeff.
From past experiences, how long does it usually take for Google to de-index pages that have the noindex and nofollow tag in the header?
Hi Jeff,
Sorry for taking so long to reply and thank you for taking the time to respond.
1. It isn't possible to password protect the demo sites for numerous.
2. The demo and live site are hosted using a load server which has multiple IP addresses, all of which change from week-to-week meaning that we cant restrict the IP addresses using the htaccess file.
3. I like the idea of adding a banner to the top of the demo site letting people know that they are not on the live site, we could also include a link through to the live site - which help usability.
Can I ask why you don't think I should issue a change of address?
Thanks,
Lloyd
The web design company tasked with building a company website have a demo site which reflects the live site - as the site is constantly being developed and needs to be tested by numerous uses, removing the demo site is not an option.
The demo site is a subdomain of the live site: demo.mysite.com
The demo site has been indexed by Google as the web design company didn't add a robots.txt file disallowing all. The demo site was indexed 3 years before I started working at the company.
My first action was to add a robots.txt file disallowing all, I then added a noindex and nofollow meta tag into the header of all files on the demo area - this has prevented any new pages from being indexed but the original pages of the site are still in Google index.
Do you think I should add the demo subdomain (demo.mysite.com) into Google Webmaster and issue a change of address pointing to the actual website (www.mysite.com) or is this going to cause problems?
All thoughts, opinions and recommendations are very much appreciated.
The site that I am optimising has a problem with duplicate pages being indexed as a result of the load balancer (which is required and set up by the hosting company).
The load balancer passes the site through to 2 different URLs:
Some how, Google have indexed 2 of the same URLs (which I was obviously hoping they wouldn't) - the first on www and the second on www2.
The hosting is a mirror image of each other (www and www2), meaning I can't upload a robots.txt to the root of www2.domain.com disallowing all. Also, I can't add a canonical script into the website header of www2.domain.com pointing the individual URLs through to www.domain.com etc.
Any suggestions as to how I can resolve this issue would be greatly appreciated!
Having unique content is not a visable option in this instance!
You mentioned that using canonical links will work to a certain degree - can you expand on this?
A company has a TLD (top-level-domain) which every single product:
The company also has subdomains (tailored to a range of products) which lists a choosen selection of the products from the TLD - sort of like a feed:
The content on the TLD & subdomain product page are exactly the same and cannot be changed - CSS and HTML is slightly differant but the content (text and images) is exactly the same!
My concern (and rightly so) is that Google will deem this to be duplicate content, therfore I'm going to have to add a rel cannonical tag into the header of all subdomain pages, pointing to the original product page on the TLD. Does this sound like the correct thing to do? Or is there a better solution?
Moving on, not only are products fed onto subdomain, there are a handfull of other domains which list the products - again, the content (text and images) is exactly the same:
Would I be best placed to add a rel cannonical tag into the header of the product pages on other domains, pointing to the original product page on the actual TLD?
Does rel cannonical work across domains?
Would the product pages with a rel cannonical tag in the header still rank?
Let me know if there is a better solution all-round!
Out of curiosity, why wouldn't you recommend solution 2?
You mentioned that you faced a similar kind of situation in the past, how did that work out? Which of the 3 solutions (or all) did you opt for?
Good advice but an IP restriction for the demo sites won't be possible on this occasion as our router throws out a range of different IP addresses and we occasionally need the sites to be viewed externally! Any other suggestions to help?
I've recently taken over an in-house role at a property auction company, they have a main site on the top-level domain (TLD) and 400+ agency sub domains!
I recently found that the web development team have a demo domain per site, which is found on a subdomain of the original domain - mirroring the site. The problem is that they have all been found and indexed by Google:
Obviously this is a problem as it is duplicate content and so on, so my question is... what is the best way to remove the demo domain / sub domains from Google's index?
We are taking action to add a noindex tag into the header (of all pages) on the individual domains but this isn't going to get it removed any time soon! Or is it?
I was also going to add a robots.txt file into the root of each domain, just as a precaution! Within this file I had intended to disallow all.
The final course of action (which I'm holding off in the hope someone comes up with a better solution) is to add each demo domain / sub domain into Google Webmaster and remove the URLs individually.
Or would it be better to go down the canonical route?