Should I issue a change of address for my demo site which has been indexed?
-
The web design company tasked with building a company website have a demo site which reflects the live site - as the site is constantly being developed and needs to be tested by numerous uses, removing the demo site is not an option.
The demo site is a subdomain of the live site: demo.mysite.com
The demo site has been indexed by Google as the web design company didn't add a robots.txt file disallowing all. The demo site was indexed 3 years before I started working at the company.
My first action was to add a robots.txt file disallowing all, I then added a noindex and nofollow meta tag into the header of all files on the demo area - this has prevented any new pages from being indexed but the original pages of the site are still in Google index.
Do you think I should add the demo subdomain (demo.mysite.com) into Google Webmaster and issue a change of address pointing to the actual website (www.mysite.com) or is this going to cause problems?
All thoughts, opinions and recommendations are very much appreciated.
-
Hi Paul,
Some great advice.
In the turn of the New Year when we have more resources we are going to take action following your plan - it is a lot more work but makes perfect sense.
Again, thank you for pointing out that I need to remove the robots.txt block so that the page commands can be discovered - I will get onto this straight away.
Thank you once again.
Lloyd
-
You've got a couple of issues here, Lloyd, that aren't going to be solved by your proposed solution. The biggest one is - after three years, you have a huge amount of dev site pages indexed, and it is quite likely those pages now have at least some incoming links. If you just de-index those pages, that link and indexing equity will be thrown away, and users will continue to arrive on the "wrong" pages for quite some time.
In my experience, to do this correctly, you've got a bit more work ahead of you.
- Move the dev site to a new subdomain.
- As soon as it's moved, add a met-robots declaration to the dev site's htaccess file to automatically add no-index/no-follow to the headers of every page. If you wish, at this point, you can also add a block in robots.txt.
- Optionally, (and most preferably) password-protect the dev site by setting up a password in htaccess, assuming there aren't a huge number of people who need to have dev site access.
- 301-redirect all the existing (old) dev site's pages to the equivalent pages on the main Domain. This can be done with a single rewrite rule in htaccess.
- Removing the robots.txt block from the current dev subdomain would also be a good idea.
- Submit the main section pages of the Dev site using the Fetch as Google/Bing bot tools in the respective Webmaster Tools to try to speed up the discovery of the 301 redirects. This should help get the redirects crawled and indexed sooner, and therefore hopefully drop out of the index a bit quicker.
The main reason for doing it this way is to preserve all the link/indexing equity that has built up over the three years. In addition, it ensures any incoming dev links or users with bookmarked links will immediately end up on the correct pages. Remember, a notice to users on the dev domain page will mean the user will have to figure out how to find the correct page on the "real" domain - not a user experience you want to inflict on your customer. By using the above solution, they can't get to the "wrong" pages in the first place.
Does that all makes sense? More work, but for a much cleaner solution, both for the user and for the search engines.
Oh, and if you decide to go the easy way and leave the current dev site in place as no-indexed, you're going to have to remove the robots.txt block. No-index and robots.txt block CANNOT be used together if the pages are already indexed. The robots block tells the search engines not to crawl the pages, which means they can never the discover the no-index commands and so will never drop the pages from the index. (That's why your existing pages haven't dropped out.)
Lemme know what you think;
Paul
-
Great, thank you for your advice Jeff.
From past experiences, how long does it usually take for Google to de-index pages that have the noindex and nofollow tag in the header?
-
Llyod-
I wouldn't recommend the change of address feature. It's meant to be used when you are migrating your main website to a new domain. Your primary (live) domain is intact; you just want to no-index / no-follow your demo site.
The change of address will also only be effective for 180 days, too.
Google recently changed their change of address tool, and Barry Schwartz of Search Engine Land walks through it:
http://searchengineland.com/google-revamps-change-address-tool-within-webmaster-tools-205829The big takeaway:
"Don't use this tool unless you are moving your primary website presence to a new address."I hope this helps!
-- Jeff -
Hi Jeff,
Sorry for taking so long to reply and thank you for taking the time to respond.
1. It isn't possible to password protect the demo sites for numerous.
2. The demo and live site are hosted using a load server which has multiple IP addresses, all of which change from week-to-week meaning that we cant restrict the IP addresses using the htaccess file.
3. I like the idea of adding a banner to the top of the demo site letting people know that they are not on the live site, we could also include a link through to the live site - which help usability.
Can I ask why you don't think I should issue a change of address?
Thanks,
Lloyd
-
This happens a lot more than you might imagine. We've even seen times where end users are placing orders on test sites because they're able to find out-of-stock items from an out-of-date demo site.
If you need to have the demo. store up and running so people can test it, I don't think you need to tell Google about the change of address. I think that the no-index / no-follow status of the site and the robots.txt should prevent Google from indexing it.
That said, there are a few more ways to protect the demo site:
1. You can password protect it, using an .htaccess file. This might not be ideal, because testing might prevent you from being able to see the whole site properly on different devices.
2. You could IP restrict the site to a small number if IP addresses, using your .htaccess file. In this way, only people who are whitelisted are able to do this. You could even get fancier, and have a form that allows testers to be able to add their IP address to the "allowed" list from a browser. Anyone else would / could be redirected to the normal site.
3. I'd also recommend putting a banner at the top of the demo site, letting people know that any orders placed there are just for testing and won't be fulfilled.
I hope this helps!
-- Jeff
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Can you check my Quran site and its technical issues?
Hi, Hope you are doing good, I saw many Quran websites working fine. I have a blog that has online Quran classes of Arabic and Tajweed related courses as well. Can you please check that if all is okay in terms of technical issues? Here is the website: https://learnquran.academy
SEO Learn Center | | ShirleyLemons0 -
Webinar: Get to Know the New Site Crawl Recording
Good morning all, Does anyone have the recording for Friday, June 9th's webinar? I was at an off-site event that day and couldn't listen in. The recording isn't listed in the moz.com/webinars section. Any help is appreciated, thank you!
SEO Learn Center | | Corporate_Synergies1 -
Backlink From Different Language Sites Will Spoil My Ranking?
Hi there! Excited to joining this community. The main question is that if I get backlinks from different language sites then that backlink will be worthy or non-worthy in the eye of search engine? I know if I target international market by presenting my content in different languages then of course it will be worthwhile for me. But what will be happened if I do not represent content in different languages on my site but get the backlines from other language sites such as: Chinese, Russian or any other? Also I have seen many sites which are getting backlinks from different language sites and they are just getting high authority. Is there any expert?
SEO Learn Center | | mianarfan3331 -
Can you have same product name and description on a local and national site at the same time without getting dinged by google?
My question is regarding a few sites we own which are all related. The local site (http://location.company.com) is a extension of the national main site (http://company.com.) We are in the process of launching these new sites as replacements to older sites we created years ago. My first question regarding SEO is it important to the search engines to have unique product content on both sites even though they promote the same products (with same pictures) and brands or can we keep the same product names and descriptions as I hope? I ask this because essentially they are the same site but with with the exception of the local extension..... We thought by re writing all the content on the main pages of both sites along with different; title tags, page title and meta descriptions that that would make the sites different enough to get away with this. Is that the case? Thank you in advance for your assistance, Jake
SEO Learn Center | | Closetstogo0 -
Site for my clients to log in and see their traffic, etc.
I have done a ton of research and I am struggling to find an easy to use, easy to understand site / tool that will allow my clients to log in and see basic information about their website traffic, rankings, referring sites, etc. in an ATTRACTIVE, EASY TO UNDERSTAND layout. Any suggestions would be greatly appreciated! Seomoz (my true favorite) Raven, webceo, and so many others are powerful tools - I don't need that, just looking for an attractive place to log in and view their stats, that's it. Thank you - have a great rest of the week! Matthew
SEO Learn Center | | Mrupp440 -
SEO Ads On Third Party Job Sites?
Hello all, I seriously doubt I will get a reply to this as I'm guessing no one has experience with it. if you do though then I would appreciate any information either via comment or PM. Basically, I need to know how I can improve the rankings of job posts within the third party sites. An example would be totaljobs.com, how can I improve the rankings of advertisements that are posted there or is it simply ranked by date and location?. When I had my eBay business, I did a lot of research on eBay SEO and discovered some important factors for ranking within the eBay website. I'm basically wanting to do the same here on job board websites for job ads posted to them. Their algorithms won't be as advanced as Google etc so I'm guessing if they do they will be open for higher keyword density levels and that kind of thing.. I've googled it quite a bit and can't find any information at all on it. Thanks.
SEO Learn Center | | LukeHutchinson0 -
Location of Rand's Webinar "Using Open Site Explorer to Uncover New Marketing Opportunities"
Aug 26th Archive of Rand's Webinar "Using Open Site Explorer to Uncover New Marketing Opportunities" I don't see this in the archive, anyone know how to get it? Thanks
SEO Learn Center | | josey0