Temporary Duplicate Sites - Do anything?
-
Hi Mozzers -
We are about to move one of our sites to Joomla. This is one of our main sites and it receives about 40 million visits a month, so the dev team is a little concerned about how the new site will handle the load.
Dev's solution, since we control about 2/3 of that traffic through our own internal email and cross promotions, is to launch the new site and not take down the old site. They would leave the old site on its current URL and make the new site something like new.sub.site.com. Traffic we control would continue to the old site, traffic that we detect as new would be re-directed to the new site. Over time (the think about 3-4 months) they would shift the traffic all to the new site, then eventually change the URL of the new site to be the URL of the old site and be done.
So this seems to be at the outset a duplicate content (whole site) issue to start with. I think the best course of action is try to preserve all SEO value on the old URL since the new URL will eventually go away and become the old URL. I could consider on the new site no-crawl/no-index tags temporarily while both sites exist, but would that be risky since that site will eventually need to take those tags off and become the only site? Rel=canonical temporarily from the new site to the old site also seems like it might not be the best answer.
Any thoughts?
-
I'm going to throw in a completely different option, because in my opinion, messing with this kind of multiple version situation is going to put your huge website at massive risk of screwed up rankings and lost traffic no matter how tricky you get.
First, I'm assuming that significant high-level load testing has been done on the dev site already. If not, that's the place to start. (I'm suspecting a Joomla site for 40 million visits a month will have lots of load-balancing in place?)
Since by all indications, the sites will be identical to the visitor, I'd suggest switching to the new site, but keeping the original site immediately available in near-line status. By setting the TTL of the DNS to a very short duration while in transition, the site could be switched back to the old version within a minute or two just by updating the DNS if something goes pear-shaped on the new site.
Then, while the old site continues to serve visitors as it always has, devs can fix whatever issue was discovered on the new site.
This would mean keeping both sites' content updated concurrently during the period of the changeover, but it sounds like you were going to have to do that anyway. There's also the small risk that some visitors would have cached DNS on their own computers and so might still get sent to the new site for a while even after the DNS had been set back to the old site, but I'd say that's a vastly smaller risk than screwing up the rankings of the whole site.
Bottom line, there are plenty of load testing/quality assurance/server over-provisioning methods for making virtually certain the new site will be able to perform before going live. Having the backup site should be a very short term insurance, rather than a long term duplication process.
That's my perspective, anyway, having done a number of large-site migrations (though certainly nothing approaching 40M visits/month)
Paul
Just for refernce, I was involved in helping after just such a major migration where the multiple sites did get indexed. It took nearly a year to rectify the situation and get the rankings/traffic/usability back in order
-
Arghhh... This sounds like a crazy situation.
If the temp site is on a temporary subdomain, you definitely don't want any of those pages seeping into the index. But 3-4 months seems like an incredibly long time to sustain this. 3-4 days seems more reasonable to handle load testing.
For example, what happens when someone links to one of the temporary pages? Unless you put a rel canonical on the page, and allow robots to crawl it, then you won't gain from that link equity.
For a shorter time period, I'd simple block all crawlers via robots.txt, add a meta "noindex, nofollow" tag to the header, and hope for the best.
But for 3-4 months, you're taking the chance of sending very confusing signals to search engines, or losing out on new link equity. You could still use the meta "noindex, nofollow" on the temp domain if you need to, and also include rel=canonical tags (these are separate directives and actually processed differently) but there's no gaurentee of a smooth transistion once you ditch the temp urls.
So... my best advice is to convince your dev team to shorten the 3-4 month time frame. Not an easy job.
-
Wow 40 million visitors a month is no joke and nothing to be taken lightly if not done right the loss of traffic could be huge.
The new site should be non indexable and you can redirect a percentage of traffic to the new site (beta.site.com) for server load testing reasons and once you determine it is stable you can move it over to the new site.
Are URLs and site structure etc remaining the same? I wouldn't change too much at once or you won't know what happened if something tanks.
-
Thanks for the response.
It might have been just an unfounded concern, based on a vague memory of something I read about rel=canonical on here, but cannot find it now.
I was just concerned that if you have site A and B and rel=canonical from B to A, then eventually get rid of A and have B take on the URL of A, that the engines might interpret this oddly and have it affect domain authority.
-
Why do you think that canonical tags won't work?
That's what I would suggest.. Those tags simply tell Google which is the authoritative site of the duplicates. If you are preserving the original domain, canonical to that one and when you make the switch nothing will change. Do keep in mind if any of your directories or file structures are altered you will want to put in redirects but it sounds like your web team knows what they're doing here.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
¿Disallow duplicate URL?
Hi comunity, thanks for answering my question. I have a problem with a website. My website is: http://example.examples.com/brand/brand1 (good URL) but i have 2 filters to show something and this generate 2 URL's more: http://example.examples.com/brand/brand1?show=true (if we put 1 filter) http://example.examples.com/brand/brand1?show=false (if we put other filter) My question is, should i put in robots.txt disallow for these filters like this: **Disallow: /*?show=***
Intermediate & Advanced SEO | | thekiller990 -
Duplicate content issue
Hello! We have a lot of duplicate content issues on our website. Most of the pages with these issues are dictionary pages (about 1200 of them). They're not exactly duplicate, but they contain a different word with a translation, picture and audio pronunciation (example http://anglu24.lt/zodynas/a-suitcase-lagaminas). What's the better way of solving this? We probably shouldn't disallow dictionary pages in robots.txt, right? Thanks!
Intermediate & Advanced SEO | | jpuzakov0 -
SEO effect of content duplication across hub of sites
Hello, I have a question about a website I have been asked to work on. It is for a real estate company which is part of a larger company. Along with several other (rival) companies it has a website of property listings which receives a feed of properties from a central hub site - so lots of potential for page, title and meta content duplication (if if isn't already occuring) across the whole network of sites. In early investigation I don't see any of these sites ranking very well at all in Google for expected search phrases. Before I start working on things that might improve their rankings, I wanted to ask some questions from you guys: 1. How would such duplication (if it is occuring) effect the SEO rankings of such sites individually, or the whole network/hub collectively? 2. Is it possible to tell if such a site has been "burnt" for SEO purposes, especially if or from any duplication? 3. If such a site or the network has been totally burnt, are there any approaches or remedies that can be made to improve the site's SEO rankings significantly, or is the only/best option to start again from scratch with a brand new site, ensuring the use of new meta descriptions and unique content? Thanks in advance, Graham
Intermediate & Advanced SEO | | gmwhite9991 -
Cross Domain duplicate content...
Does anyone have any experience with this situation? We have 2 ecommerce websites that carry 90% of the same products, with mostly duplicate product descriptions across domains. We will be running some tests shortly. Question 1: If we deindex a group of product pages on Site A, should we see an increase in ranking for the same products on Site B? I know nothing is certain, just curious to hear your input. The same 2 domains have different niche authorities. One is healthcare products, the other is general merchandise. We've seen this because different products rank higher on 1 domain or the other. Both sites have the same Moz Domain Authority (42, go figure). We are strongly considering cross domain canonicals. Question 2 Does niche authority transfer with a cross domain canonical? In other words, for a particular product, will it rank the same on both domains regardless of which direction we canonical? Ex: Site A: Healthcare Products, Site B: General Merchandise. I have a health product that ranks #15 on site A, and #30 on site B. If I use rel=canonical for this product on site B pointing at the same product on Site A, will the ranking be the same if I use Rel=canonical from Site A to Site B? Again, best guess is fine. Question 3: These domains have similar category page structures, URLs, etc, but feature different products for a particular category. Since the pages are different, will cross domain canonicals be honored by Google?
Intermediate & Advanced SEO | | AMHC1 -
Is This Considered Duplicate Content?
My site has entered SEO hell and I am not sure how to fix it. Up until 18 months ago I had tremendous success on Google and Bing and now my website appears below my Facebook page for the term "Direct Mail Raleigh." What makes it even more frustrating is my competitors have done no SEO and they are dominating this keyword. I thought that the issue was due to harmful inbound links and two months ago I disavowed ones that were clearly spam. Somehow my site has actually gone down! I have a blog that I have updated infrequently and I do not know if it I am getting punished for duplicate content. On Google Webmaster Tools it says I have 279 crawled and indexed pages. Yesterday when I ran the MOZ crawl check I was amazed to find 1150 different webpages on my site. Despite the fact that it does not appear on the webmaster tools I have three different webpages due to the format that the Wordpress blog was created: "http://www.marketplace-solutions.com/report/part2leadershi/", "http://www.marketplace-solutions.com/report/page/91/" and "http://www.marketplace-solutions.com/report/category/competent-leadership/page/3/" What does not make sense to me is why Google only indexed 279 webpages AND why MOZ did not identify these three webpages as duplicate content with the Crawl Test Tool. Does anyone have any ideas? Would it be as easy as creating a massive robot.txt file and just putting 2 of the 3 URLs in that file? Thank you for your help.
Intermediate & Advanced SEO | | DR700950 -
Why does this site rank above us?
We own www.discountbannerprinting.co.uk and over the last 8 months have built some decent guest post, charity and customer links but still we seem to be beaten on good words such as banners, banner, vinyl banner, pvc banner etc by this website www-signfirm.com we just cannot figure out how this is happening and would be very grateful if someone with great wisdom could give us an in-site into why this is happening and we would be very grateful..
Intermediate & Advanced SEO | | BobAnderson0 -
Duplicate content for images
On SEOmoz I am getting duplicate errors for my onsite report. Unfortunately it does not specify what that content is... We are getting these errors for our photo gallery and i am assuming that the reason is some of the photos are listed in multiple categories. Can this be the problem? what else can it be? how can we resolve these issues?
Intermediate & Advanced SEO | | SEODinosaur0 -
Should I do something about this duplicate content? If so, what?
On our real estate site we have our office listings displayed. The listings are generated from a scraping script that I wrote. As such, all of our listings have the exact same description snippet as every other agent in our office. The rest of the page consists of site-wide sidebars and a contact form. The title of the page is the address of the house and so is the H1 tag. Manually changing the descriptions is not an option. Do you think it would help to have some randomly generated stuff on the page such as "similar listings"? Any other ideas? Thanks!
Intermediate & Advanced SEO | | MarieHaynes0