Should I issue a change of address for my demo site which has been indexed?
-
The web design company tasked with building a company website have a demo site which reflects the live site - as the site is constantly being developed and needs to be tested by numerous uses, removing the demo site is not an option.
The demo site is a subdomain of the live site: demo.mysite.com
The demo site has been indexed by Google as the web design company didn't add a robots.txt file disallowing all. The demo site was indexed 3 years before I started working at the company.
My first action was to add a robots.txt file disallowing all, I then added a noindex and nofollow meta tag into the header of all files on the demo area - this has prevented any new pages from being indexed but the original pages of the site are still in Google index.
Do you think I should add the demo subdomain (demo.mysite.com) into Google Webmaster and issue a change of address pointing to the actual website (www.mysite.com) or is this going to cause problems?
All thoughts, opinions and recommendations are very much appreciated.
-
Hi Paul,
Some great advice.
In the turn of the New Year when we have more resources we are going to take action following your plan - it is a lot more work but makes perfect sense.
Again, thank you for pointing out that I need to remove the robots.txt block so that the page commands can be discovered - I will get onto this straight away.
Thank you once again.
Lloyd
-
You've got a couple of issues here, Lloyd, that aren't going to be solved by your proposed solution. The biggest one is - after three years, you have a huge amount of dev site pages indexed, and it is quite likely those pages now have at least some incoming links. If you just de-index those pages, that link and indexing equity will be thrown away, and users will continue to arrive on the "wrong" pages for quite some time.
In my experience, to do this correctly, you've got a bit more work ahead of you.
- Move the dev site to a new subdomain.
- As soon as it's moved, add a met-robots declaration to the dev site's htaccess file to automatically add no-index/no-follow to the headers of every page. If you wish, at this point, you can also add a block in robots.txt.
- Optionally, (and most preferably) password-protect the dev site by setting up a password in htaccess, assuming there aren't a huge number of people who need to have dev site access.
- 301-redirect all the existing (old) dev site's pages to the equivalent pages on the main Domain. This can be done with a single rewrite rule in htaccess.
- Removing the robots.txt block from the current dev subdomain would also be a good idea.
- Submit the main section pages of the Dev site using the Fetch as Google/Bing bot tools in the respective Webmaster Tools to try to speed up the discovery of the 301 redirects. This should help get the redirects crawled and indexed sooner, and therefore hopefully drop out of the index a bit quicker.
The main reason for doing it this way is to preserve all the link/indexing equity that has built up over the three years. In addition, it ensures any incoming dev links or users with bookmarked links will immediately end up on the correct pages. Remember, a notice to users on the dev domain page will mean the user will have to figure out how to find the correct page on the "real" domain - not a user experience you want to inflict on your customer. By using the above solution, they can't get to the "wrong" pages in the first place.
Does that all makes sense? More work, but for a much cleaner solution, both for the user and for the search engines.
Oh, and if you decide to go the easy way and leave the current dev site in place as no-indexed, you're going to have to remove the robots.txt block. No-index and robots.txt block CANNOT be used together if the pages are already indexed. The robots block tells the search engines not to crawl the pages, which means they can never the discover the no-index commands and so will never drop the pages from the index. (That's why your existing pages haven't dropped out.)
Lemme know what you think;
Paul
-
Great, thank you for your advice Jeff.
From past experiences, how long does it usually take for Google to de-index pages that have the noindex and nofollow tag in the header?
-
Llyod-
I wouldn't recommend the change of address feature. It's meant to be used when you are migrating your main website to a new domain. Your primary (live) domain is intact; you just want to no-index / no-follow your demo site.
The change of address will also only be effective for 180 days, too.
Google recently changed their change of address tool, and Barry Schwartz of Search Engine Land walks through it:
http://searchengineland.com/google-revamps-change-address-tool-within-webmaster-tools-205829The big takeaway:
"Don't use this tool unless you are moving your primary website presence to a new address."I hope this helps!
-- Jeff -
Hi Jeff,
Sorry for taking so long to reply and thank you for taking the time to respond.
1. It isn't possible to password protect the demo sites for numerous.
2. The demo and live site are hosted using a load server which has multiple IP addresses, all of which change from week-to-week meaning that we cant restrict the IP addresses using the htaccess file.
3. I like the idea of adding a banner to the top of the demo site letting people know that they are not on the live site, we could also include a link through to the live site - which help usability.
Can I ask why you don't think I should issue a change of address?
Thanks,
Lloyd
-
This happens a lot more than you might imagine. We've even seen times where end users are placing orders on test sites because they're able to find out-of-stock items from an out-of-date demo site.
If you need to have the demo. store up and running so people can test it, I don't think you need to tell Google about the change of address. I think that the no-index / no-follow status of the site and the robots.txt should prevent Google from indexing it.
That said, there are a few more ways to protect the demo site:
1. You can password protect it, using an .htaccess file. This might not be ideal, because testing might prevent you from being able to see the whole site properly on different devices.
2. You could IP restrict the site to a small number if IP addresses, using your .htaccess file. In this way, only people who are whitelisted are able to do this. You could even get fancier, and have a form that allows testers to be able to add their IP address to the "allowed" list from a browser. Anyone else would / could be redirected to the normal site.
3. I'd also recommend putting a banner at the top of the demo site, letting people know that any orders placed there are just for testing and won't be fulfilled.
I hope this helps!
-- Jeff
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Backlink From Different Language Sites Will Spoil My Ranking?
Hi there! Excited to joining this community. The main question is that if I get backlinks from different language sites then that backlink will be worthy or non-worthy in the eye of search engine? I know if I target international market by presenting my content in different languages then of course it will be worthwhile for me. But what will be happened if I do not represent content in different languages on my site but get the backlines from other language sites such as: Chinese, Russian or any other? Also I have seen many sites which are getting backlinks from different language sites and they are just getting high authority. Is there any expert?
SEO Learn Center | | mianarfan3331 -
What are you doing to change their stars?
I read on PCWorld a piece by John Rebeiro (@JohnRebeiro) about Google's employee diversity. In it, Google in the US is 70% male and 61% white. There is more data and it's worth a read. My question for those of you who may employ more than a 2 to 3 people is are you actively engaged in doing anything to change the stars of young women, minorities, lower income children, etc.? Why do I ask? Because early on in my Mozziness I saw someone ask for some help with a site around children and autism. From Rand to every mozzer on Q&A there was an outpouring of help. I was truly moved. I would love to learn from others who are further along in these types of efforts.
SEO Learn Center | | RobertFisher
Google does make an effort in that they have RISE awards to assist with those who are working to promote STEM (science, technology, engineering, mathematics) with girls and minorities. These awards are from $10K to $100K to help expand these types of programs. We have had paid interns for some time now and we get as much or more than we give from that effort (boastful disclosure: Our head of development today was our first intern!) We put on a one week Code Camp (go to our site/code-camp) a few months ago and it was amazing - for me and for our team. But, to see kids eyes open up as they realize they have found something they never even thought they would have an interest in is mind blowing. We decided we would give a $25 gift card to Best Buy for the student who was most engaged and pushed the most to learn more, etc. That winner had never thought of coding as something SHE would have an interest in and she was an unbelievable sponge. She pushed our dev staff to give her more and more and it was truly fun and rewarding for them. Yes, we had to spend some time and money, but the break from the day to day of a marketing firm for our team truly invigorated them. I just want to know what others might be doing in the areas of trying get more diversity in the ranks of those who work in our world? It would be great to share ideas and resources. I will tell you that if anyone needs tips, what we learned, etc. I would be more than happy to share all.3 -
I changed the {Site Name} of our domain by capitalizing a letter. 48 hours later our SEO rankings are bad. How long does it take to rebound our prior rankings. The {site name} was at the end of every page title.
I changed the {Site Name} of our domain slightly and capitalized a letter. The impact 48 hours later on SEO is really, really bad. How long does it take to recapture our prior rankings and should I change the site name back to the original? The {site name} was at the end of every page title. Thanks for any help/advice on this. We worked so hard to get our business on the first page for many keywords and "POOF" we are gone now.
SEO Learn Center | | LinckB0 -
Index Page Dropped From Rankings
On average my site was getting ranked 3rd for "Tucson DUI Lawyer", Best Tucson DUI Lawyer, Tucson DUI attorney and etc. Recently my website fell for all of these keywords and I thought it is the best time to launch the new website. The problem is that my internal pages show up for some of the keywords on the third or fourth page but my index page doesn't show for any of them. Does it have anything to do with the site structure ? I have a lot of news excerpts on the homepage with links to the main posts. Here is the link: http://www.michaelharwinlaw.com/ Any help is sincerely appreciated.
SEO Learn Center | | yasinyaqoobi0 -
How much dev knowledge as an seo do I need to know in order to make small changes on websites?
Hi mozzers I am considering learning some dev just to be able to make minor changes on websites. I love seo but will never want to be a programmer. I get sometimes frustrated that I have to request my dev colleague to make minor changes such as adding a small piece of codes to a website for verification purposes or adding rich snippet or modifying the ga code to get more out of it. I know some HTML and understands the core elements and attributes. My first question would be: what should I learn without learning A-Z of programming? My guess would be some php and? My second question: based on your answer where can I learn about these programs? Any interactive learning services such as team treehouse ? Any online courses? Thanks!
SEO Learn Center | | Ideas-Money-Art1 -
Top Domain Authority Sites
Is there a list like the IR500, but for the top domain authority sites? If not, could you build one?
SEO Learn Center | | adam-1869630 -
Location of Rand's Webinar "Using Open Site Explorer to Uncover New Marketing Opportunities"
Aug 26th Archive of Rand's Webinar "Using Open Site Explorer to Uncover New Marketing Opportunities" I don't see this in the archive, anyone know how to get it? Thanks
SEO Learn Center | | josey0