My automated build system is creating a duplicate website
-
Because of the tools my company is using for CI/CD (A CI/CD pipeline helps you automate steps in your software delivery process, such as initiating code builds, running automated tests, and deploying to a staging or production environment.) an extra URL is generated. The canonical for the generated site is that of our main website, but other than that it is the same website.
- Could this new URL compete with our website?
- Will Google count it against us since it is the same content BUT with canonical (it is not noindex-ed)?
- Does it matter?
- Surely others are using this method?
Answers/thoughts will be greatly appreciated. Thank you.
-
Do you have any control over the CI/CD pipeline URL?
If you control the domain enough so that you can be one to have validated and searched console them by all means. But it does not seem like you have the ability to control domain?
my correct?
https://support.google.com/webmasters/answer/7440203?hl=en
If the domain is 3ed party domain then you must trust the third-party or if you control the domain of pages which links or third-party domain URLs are embedded on you can add noindex nofollow
https://www.deepcrawl.com/blog/best-practice/noindex-disallow-nofollow/
I hope that helps,
Tom
-
Unfortunately, since URL is generated from the original site, I cannot change the robots.txt. It uses the same one as the main site. That would exclude adding a noindex meta tag, as well. Any other ideas?
Is there a way to add the duplicate URL to search console & tell google not to crawl?
Thank you.
-
I understand using CI cool
i agree get the bad content being made by CI blocked ASAP
“have an extra URL is generated. The canonical for the generated site is that of our main website, but other than that it is the same website.”
but it’s not the same content being made that will hurt you unless you’re pointing the canonicals to a similar page (get the automated content off your domain)
Remember to add using self pointing canonicals on the good pages you want to be indexed by Google or Search Engines
Hope this is of help,
Tom
-
To answer your questions:
- Technically it could compete with your current site as it's on its own domain, in reality, it's unlikely as you're canonicalizing the pages back to its original and making sure that the content itself through that way is attributed to your original site.
- What I would recommend is excluding the CI/CD site from the engines, through a robots.txt or a similar technique. That way you're making sure that the staging site itself isn't being crawled at all. In the end, I'd say there's very little upside of having that be the case currently.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Specific Industry Website Conversion Rates: Lighting
Hi All, There's loads of info around on general retail conversion rates, but does anyone have any experience with online lighting shops and typical conversion rates? This is a highly price driven shopper, and from my experience so far they bounce around looking for the best price... We've recently taken ownership of this new site, and I'm not sure I can relate general metrics to this site... although there is lots of work to do on here! Cheers in advance.
Reporting & Analytics | | b4cab0 -
Pages with Duplicate Page Content
Hi Just started use the Moz and got an analytics report today! There about 104 duplicate pages apparently, the problem is that they are not duplicates, but just the way the page has been listed with a description! The site is an Opencart and every page as got the name of the site followed by the product name for the page! How do you correct this issue?? Thank for your help
Reporting & Analytics | | DRSMPR1 -
Hi, my website suddenly fell to the sixth page. All of the keywords. Is this sandbox?
www.enakliyat.com.tr this is my website when, i write the google "enakliyat" i found my website in te sixth page. Two days ago my website was on the first in google. Do not understand what it is.
Reporting & Analytics | | iskq0 -
Can you help me figure out what happened to my website search results in Google?
On or about the 24th of April I noticed an abrupt decrease in traffic to my website:
Reporting & Analytics | | rdominey
http://www.getyourphotosoncanvas.com Sorry this might be long but I’m trying to be as thorough as possible. I thought that I had been hacked, a virus, maybe penalized by Google I don’t know what ? I submitted a reconsideration request to Google and they responded with the following: Reconsideration request for http://www.getyourphotosoncanvas.com/: No manual spam actions found
May 10, 2012
Dear site owner or webmaster of http://www.getyourphotosoncanvas.com/,
We received a request from a site owner to reconsider http://www.getyourphotosoncanvas.com/ for compliance with Google's Webmaster Guidelines. - - - - - -
We reviewed your site and found no manual actions by the webspam team that might affect your site's ranking in Google. There's no need to file a reconsideration request for your site, because any ranking issues you may be experiencing are not related to a manual action taken by the webspam team.
Google Search Quality Team I have ran all kinds of web crawl tests, Google webmaster, talked with SEO “Experts” and still can not figure out what is happening. I decided to use a couple of SEOmoz tools to try to help me explain what is happening. I figured that if I could take a very specific and unique KeyPhrase and run it on a specific page that I might be able to better explain what is happening. Basically, We appear to be no longer searchable by key words or phrases on google?
Here is an example:
Key Phrase: Free Services to Help Improve Your Photos on Canvas
Website: http://www.getyourphotosoncanvas.com/free-photo-canvas-retouching
Attached are some screen shots of the actual search results on Bing, Yahoo and Google along with the ranking tool results from SEOmoz and the on page grade for the key phrase.
Anybody got any Ideas? I am hurting; the internet and Google search is about 40% of by business. http://www.getyourphotosoncanvas.com/wp-content/uploads/2012/05/Bing-Free-Services.jpg http://www.getyourphotosoncanvas.com/wp-content/uploads/2012/05/Yahoo-Free-Services.jpg http://www.getyourphotosoncanvas.com/wp-content/uploads/2012/05/Google-Free-Services.jpg http://www.getyourphotosoncanvas.com/wp-content/uploads/2012/05/SEOmoz-Ranking.jpg http://www.getyourphotosoncanvas.com/wp-content/uploads/2012/05/SEOmoz-Report-Card.jpg [" target="_blank">iframe>](<iframe class=) Bing-Free-Services.jpg Yahoo-Free-Services.jpg Google-Free-Services.jpg SEOmoz-Ranking.jpg SEOmoz-Report-Card.jpg0 -
Dramatic Increase in referrals from own website
The past few weeks I've been wracking my brain to figure out why on earth my branded searches could be dropping off at 30-40%. Well today, I realize I've had a dramatic increase in referrals from my own site (the non www verison). I'm talking 150 in March of last year to 5k in March of this year drastic. My 301 redirects haven't changed as far as I know -- I've had them set to redirect from the non www. to the www. for at least a year or two. I'm assuming visitors from search engines are somehow getting the non www version and the redirect is attributing the traffic to referrals instead of search. The drop in search traffic and the increase in referral traffic fall on the same day. Does that sound right/possible? If so, how do I fix this? My traffic stats for two clients are all screwy because of this. I want to make sure whatever solution I implement won't hurt my search traffic numbers any more 🙂 Has anyone else seen this happen recently? I could imagine an anomaly with one site but I find it odd that it could be two client sites. I still have some others to check. Thanks in advance! Leslie
Reporting & Analytics | | LeslieVS0 -
Duplicate content? Split URLs? I don't know what to call this but it's seriously messing up my Google Analytics reports
Hi Friends, This issue is crimping my analytics efforts and I really need some help. I just don't trust the analytics data at this point. I don't know if my problem should be called duplicate content or what, but the SEOmoz crawler shows the following URLS (below) on my nonprofit's website. These are all versions of our main landing pages, and all google analytics data is getting split between them. For instance, I'll get stats for the /camp page and different stats for the /camp/ page. In order to make my report I need to consolidate the 2 sets of stats and re-do all the calculations. My CMS is looking into the issue and has supposedly set up redirects to the pages w/out the trailing slash, but they said that setting up the "ref canonical" is not relevant to our situation. If anyone has insights or suggestions I would be grateful to hear them. I'm at my wit's end (and it was a short journey from my wit's beginning ...) Thanks. URL www.enf.org/camp www.enf.org/camp/ www.enf.org/foundation www.enf.org/foundation/ www.enf.org/Garden www.enf.org/garden www.enf.org/Hante_Adventures www.enf.org/hante_adventures www.enf.org/hante_adventures/ www.enf.org/oases www.enf.org/oases/ www.enf.org/outdoor_academy www.enf.org/outdoor_academy/
Reporting & Analytics | | DMoff0 -
Mobile Website Analytics Code and Button Tracking / Event Tracking
How to track the action on the mobile version. Action by pressing the "add comment" this code: onclick = "_gaq.push (['_trackEvent', 'comments', 'pressed'])
Reporting & Analytics | | meteorr
- Not suitable for mobile version Please help.0 -
I made 18 websites and the traffic keeps going down over 3 months
I made 18 websites, and have used a analytics web app called piwik. You can google it, but basically it is like google analytics. I have done nothing for the websites no links, no updates. I did do the onpage optimization extremely well. At first I had daily traffic over all the websites at about 200, then like a month went by and it was at 100, then another month has gone by it is hovering around 30 visits -- This is total traffic across all the websites. In addition my websites were ranking much better and alot of them were coming up together in the results in a single google query, now this is no longer true, only one or maybe two come for the same google query and they come up lower in the serp ranking ie. before it was 1st place now 3rd for example, so traffic has decreased respectively. Anybody can tell me what I can do, to regain the positions and traffic I had before.
Reporting & Analytics | | mickey110