My automated build system is creating a duplicate website
-
Because of the tools my company is using for CI/CD (A CI/CD pipeline helps you automate steps in your software delivery process, such as initiating code builds, running automated tests, and deploying to a staging or production environment.) an extra URL is generated. The canonical for the generated site is that of our main website, but other than that it is the same website.
- Could this new URL compete with our website?
- Will Google count it against us since it is the same content BUT with canonical (it is not noindex-ed)?
- Does it matter?
- Surely others are using this method?
Answers/thoughts will be greatly appreciated. Thank you.
-
Do you have any control over the CI/CD pipeline URL?
If you control the domain enough so that you can be one to have validated and searched console them by all means. But it does not seem like you have the ability to control domain?
my correct?
https://support.google.com/webmasters/answer/7440203?hl=en
If the domain is 3ed party domain then you must trust the third-party or if you control the domain of pages which links or third-party domain URLs are embedded on you can add noindex nofollow
https://www.deepcrawl.com/blog/best-practice/noindex-disallow-nofollow/
I hope that helps,
Tom
-
Unfortunately, since URL is generated from the original site, I cannot change the robots.txt. It uses the same one as the main site. That would exclude adding a noindex meta tag, as well. Any other ideas?
Is there a way to add the duplicate URL to search console & tell google not to crawl?
Thank you.
-
I understand using CI cool
i agree get the bad content being made by CI blocked ASAP
“have an extra URL is generated. The canonical for the generated site is that of our main website, but other than that it is the same website.”
but it’s not the same content being made that will hurt you unless you’re pointing the canonicals to a similar page (get the automated content off your domain)
Remember to add using self pointing canonicals on the good pages you want to be indexed by Google or Search Engines
Hope this is of help,
Tom
-
To answer your questions:
- Technically it could compete with your current site as it's on its own domain, in reality, it's unlikely as you're canonicalizing the pages back to its original and making sure that the content itself through that way is attributed to your original site.
- What I would recommend is excluding the CI/CD site from the engines, through a robots.txt or a similar technique. That way you're making sure that the staging site itself isn't being crawled at all. In the end, I'd say there's very little upside of having that be the case currently.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Big drop in organic traffic after moving the website-should we still do 301 redirects?
Hi,
Reporting & Analytics | | martin1970
We have a website that got redesigned with new urls in Jan 31, 2018. Since then our SEO traffic has gone down big time and to never recover. We did not do any 301 redirects back then (very stupid I know but I was not in charge then). So my question is would it be beneficial to 301 redirect old urls that were once ranked but now have all 404 errors or is it too late to do these 301 to gain any benefits? If a page that was once ranked and then have a 404 error, how long does google keep that 404 page in their database? I have heard information saying that although the page is a 404 it may still be indexed in their backend for some time and then it completely drops off all together. If so do you know how long time they would keep those 404 in their database? The old urls may have had good backlinks pointed to them because the organic traffic was good back then. So I wonder if doing 301 right now would help send some link juice over to the new urls? Or would this be a complete waste of time? Cheers Martin1 -
How to create goal for Events?
Hi All, I have created event via google tag manager for my ecommerce site. That events are button clicks. Now I want to know after clicking that event does that customer reached to Thank You Page or not? How can I do that? which type of goal I have to create? Thanks!
Reporting & Analytics | | pragnesh96390 -
Goal Tracking In Analytics On Separate Ordering Website
I have a question for any Google Analytics wizards out there. We have two clients that have a similar complication when it comes to tracking goal conversions in Analytics. Basically, all the conversion actions we want to track occur on a separate website; either an iframe embedded on the page or through an entirely different ordering website. Trick is, Analytics sees the source for all these conversions as referrals from the main site. We'd like to get visibility back to the original source/medium that brought visitors to the site before they converted. Anyone have a suggestion for making that happen?
Reporting & Analytics | | fivefifty0 -
OSE shows URLs redirecting to our custom created error page, is this a problem?.
When I check the link metrics for my product pages in OSE, it shows a message saying that the page redirects to our custom error page. This page was recently created to display when there is an error with the website. Do I need to be concerned that OSE is seeing all product pages as redirecting to this error page? Will it affect page authority etc,? I have attached a screen shot of the message that OSE displays for reference. YWbpM.jpg
Reporting & Analytics | | pugh0 -
How to create an advanced segmentation for Google+?
Hey Mozzers, I have a question regarding Google Analytics. The problem: I have created advanced segments for Social Media: Facebook, Twitter, Youtube, etc. They all work fine... Now I am trying to create something similar for Google+ (since soon we are launching a campaign for it and I want to be able to check the evolution fast and effective). My advanced segments look like: Include -> Source -> Matching RegExp -> (Facebook|Youtube|Twitter) How should the Google+ be inserted into the group? I know this question might be easy for some advanced Analytics users but I am stuck at this point. Any help appreciated! Thanks in advance! Istvan
Reporting & Analytics | | Keszi0 -
Mobile Website Analytics Code and Button Tracking / Event Tracking
How to track the action on the mobile version. Action by pressing the "add comment" this code: onclick = "_gaq.push (['_trackEvent', 'comments', 'pressed'])
Reporting & Analytics | | meteorr
- Not suitable for mobile version Please help.0 -
How do you add Facebook Insights to Multiple Websites?
If I have 4 top-level domain websites all for the same corporate company but with different focuses (ie/ engineering, technology etc.) and I want to add Facebook insights, should I add the same meta tag to all websites or should I create separate meta tags? If I create seperate ones, the issue is that we have one main facebook page that has multiple admins that I would like to give access to but it looks like you can only associate one website to one facebook page. So from what I can tell, if I were to create separate ones, I would have to setup fake facebook pages or fake profiles in order to get different facebook insights meta tags? I'm hoping there is a better way... or maybe I should just put the same meta tag code on all 4 websites? Thanks for your help!
Reporting & Analytics | | randstadsocial0 -
Sub-category considered duplicate content?
Hello, My craw diagnostics from the PRO account is telling me that the following two links have duplicate content and duplicate title tag: http://www.newandupcoming.com/new-blu-ray-releases (New Blu-ray Releases) http://www.newandupcoming.com/new-blu-ray-releases/action-adventure (New Action & Adventure Releases | Blu-ray) I am really new to the SEO world so I am stuck trying to figure out the best solution for this issue. My question is how should I fix this issue. I guess I can put canonical tag on all sub-categories but I was worried that search engines would not craw the sub-categories and index potentially valuable pages. Thanks for all the help.
Reporting & Analytics | | hirono0