Temporary Duplicate Sites - Do anything?

[Kenn_Gold]

Hi Mozzers -

We are about to move one of our sites to Joomla. This is one of our main sites and it receives about 40 million visits a month, so the dev team is a little concerned about how the new site will handle the load.

Dev's solution, since we control about 2/3 of that traffic through our own internal email and cross promotions, is to launch the new site and not take down the old site. They would leave the old site on its current URL and make the new site something like new.sub.site.com.  Traffic we control would continue to the old site, traffic that we detect as new would be re-directed to the new site. Over time (the think about 3-4 months) they would shift the traffic all to the new site, then eventually change the URL of the new site to be the URL of the old site and be done.

So this seems to be at the outset a duplicate content (whole site) issue to start with.  I think the best course of action is try to preserve all SEO value on the old URL since the new URL will eventually go away and become the old URL. I could consider on the new site no-crawl/no-index tags temporarily while both sites exist, but would that be risky since that site will eventually need to take those tags off and become the only site?  Rel=canonical temporarily from the new site to the old site also seems like it might not be the best answer.

Any thoughts?

[ThompsonPaul]

I'm going to throw in a completely different option, because in my opinion, messing with this kind of multiple version situation is going to put your huge website at massive risk of screwed up rankings and lost traffic no matter how tricky you get.

First, I'm assuming that significant high-level load testing has been done on the dev site already. If not, that's the place to start. (I'm suspecting a Joomla site for 40 million visits a month will have lots of load-balancing in place?)

Since by all indications, the sites will be identical to the visitor, I'd suggest switching to the new site, but keeping the original site immediately available in near-line status. By setting the TTL of the DNS to a very short duration while in transition, the site could be switched back to the old version within a minute or two just by updating the DNS if something goes pear-shaped on the new site.

Then, while the old site continues to serve visitors as it always has, devs can fix whatever issue was discovered on the new site.

This would mean keeping both sites' content updated concurrently during the period of the changeover, but it sounds like you were going to have to do that anyway. There's also the small risk that some visitors would have cached DNS on their own computers and so might still get sent to the new site for a while even after the DNS had been set back to the old site, but I'd say that's a vastly smaller risk than screwing up the rankings of the whole site.

Bottom line, there are plenty of load testing/quality assurance/server over-provisioning methods for making virtually certain the new site will be able to perform before going live. Having the backup site should be a very short term insurance, rather than a long term duplication process.

That's my perspective, anyway, having done a number of large-site migrations (though certainly nothing approaching 40M visits/month)

Paul

Just for refernce, I was involved in helping after just such a major migration where the multiple sites did get indexed. It took nearly a year to rectify the situation and get the rankings/traffic/usability back in order

[Cyrus-Shepard]

Arghhh... This sounds like a crazy situation.

If the temp site is on a temporary subdomain, you definitely don't want any of those pages seeping into the index. But 3-4 months seems like an incredibly long time to sustain this. 3-4 days seems more reasonable to handle load testing.

For example, what happens when someone links to one of the temporary pages? Unless you put a rel canonical on the page, and allow robots to crawl it, then you won't gain from that link equity.

For a shorter time period, I'd simple block all crawlers via robots.txt, add a meta "noindex, nofollow" tag to the header, and hope for the best.

But for 3-4 months, you're taking the chance of sending very confusing signals to search engines, or losing out on new link equity. You could still use the meta "noindex, nofollow" on the temp domain if you need to, and also include rel=canonical tags (these are separate directives and actually processed differently) but there's no gaurentee of a smooth transistion once you ditch the temp urls.

So... my best advice is to convince your dev team to shorten the 3-4 month time frame.  Not an easy job.

[irvingw]

Wow 40 million visitors a month is no joke and nothing to be taken lightly if not done right the loss of traffic could be huge.

The new site should be non indexable and you can redirect a percentage of traffic to the new site (beta.site.com) for server load testing reasons and once you determine it is stable you can move it over to the new site.

Are URLs and site structure etc remaining the same? I wouldn't change too much at once or you won't know what happened if something tanks.

[Kenn_Gold]

Thanks for the response.

It might have been just an unfounded concern, based on a vague memory of something I read about rel=canonical on here, but cannot find it now.

I was just concerned that if you have site A and B and rel=canonical from B to A, then eventually get rid of A and have B take on the URL of A, that the engines might interpret this oddly and have it affect domain authority.

[jesse-landry]

Why do you think that canonical tags won't work?

That's what I would suggest.. Those tags simply tell Google which is the authoritative site of the duplicates. If you are preserving the original domain, canonical to that one and when you make the switch nothing will change. Do keep in mind if any of your directories or file structures are altered you will want to put in redirects but it sounds like your web team knows what they're doing here.