My automated build system is creating a duplicate website
-
Because of the tools my company is using for CI/CD (A CI/CD pipeline helps you automate steps in your software delivery process, such as initiating code builds, running automated tests, and deploying to a staging or production environment.) an extra URL is generated. The canonical for the generated site is that of our main website, but other than that it is the same website.
- Could this new URL compete with our website?
- Will Google count it against us since it is the same content BUT with canonical (it is not noindex-ed)?
- Does it matter?
- Surely others are using this method?
Answers/thoughts will be greatly appreciated. Thank you.
-
Do you have any control over the CI/CD pipeline URL?
If you control the domain enough so that you can be one to have validated and searched console them by all means. But it does not seem like you have the ability to control domain?
my correct?
https://support.google.com/webmasters/answer/7440203?hl=en
If the domain is 3ed party domain then you must trust the third-party or if you control the domain of pages which links or third-party domain URLs are embedded on you can add noindex nofollow
https://www.deepcrawl.com/blog/best-practice/noindex-disallow-nofollow/
I hope that helps,
Tom
-
Unfortunately, since URL is generated from the original site, I cannot change the robots.txt. It uses the same one as the main site. That would exclude adding a noindex meta tag, as well. Any other ideas?
Is there a way to add the duplicate URL to search console & tell google not to crawl?
Thank you.
-
I understand using CI cool
i agree get the bad content being made by CI blocked ASAP
“have an extra URL is generated. The canonical for the generated site is that of our main website, but other than that it is the same website.”
but it’s not the same content being made that will hurt you unless you’re pointing the canonicals to a similar page (get the automated content off your domain)
Remember to add using self pointing canonicals on the good pages you want to be indexed by Google or Search Engines
Hope this is of help,
Tom
-
To answer your questions:
- Technically it could compete with your current site as it's on its own domain, in reality, it's unlikely as you're canonicalizing the pages back to its original and making sure that the content itself through that way is attributed to your original site.
- What I would recommend is excluding the CI/CD site from the engines, through a robots.txt or a similar technique. That way you're making sure that the staging site itself isn't being crawled at all. In the end, I'd say there's very little upside of having that be the case currently.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Shall i index double pages of my website as compared to my competitors?
a:my competitors has indexed 10 pages (checked it with site:abcd.com and found 10 results) b:what if i index 20 pages of my site and create a lot of content which is also better than my competitors who will have the edge?
Reporting & Analytics | | calvinkj0 -
I have had a huge increase in direct traffic to our website but not sure why this suddenly happened? (no promos during this time period)
I have had a huge increase in direct traffic to our website but not sure why this suddenly happened? (no promos during this time period), traffic up 200%+ according to Google Analytics
Reporting & Analytics | | Julia_a1a1 -
Is there an easy way to switch hundreds of websites to https in GSC?
My company has hundreds of websites setup in Google Search Console but will soon be moving them all to secure domains. Is there an easy way to make the switch in GSC or do we have to change the address one by one?
Reporting & Analytics | | MJTrevens0 -
How to track mobile actions on my website
Hi, i have a website that have, as many others, a mobile version. I want to know if there exist a way to track the mobile user experience, for example, how many times scroll down the page, where touch the screen, and actions that only have sense with mobile visitors. what do you think? Thanks, Juan Ignacio
Reporting & Analytics | | NachoRetta0 -
Goal Tracking In Analytics On Separate Ordering Website
I have a question for any Google Analytics wizards out there. We have two clients that have a similar complication when it comes to tracking goal conversions in Analytics. Basically, all the conversion actions we want to track occur on a separate website; either an iframe embedded on the page or through an entirely different ordering website. Trick is, Analytics sees the source for all these conversions as referrals from the main site. We'd like to get visibility back to the original source/medium that brought visitors to the site before they converted. Anyone have a suggestion for making that happen?
Reporting & Analytics | | fivefifty0 -
What should I expect after completely redoing a website?
Hello Forum, My team and I have completely redone an eCommerce website, which offers new SEO-friendly URLs. We have also taken care to optimize our product and category pages for specific keywords, so as to keep our regular traffic and increase the long-tail. In the old un-optimized website, many of its Google search results simply took users to the homepage, thus the homepage has a lot of external link juice. Our new site is optimized so that users are taken to the page most relevant to their query. For example (we are a yoga site), with the old site a search for "yoga bolsters" would take visitors to the homepage. Now the search results take the users to the yoga bolster page. These new pages don't have the amount of external links as the new site and I've noticed a decrease in our site's average rank in Google Webmaster Tools 1.Given our new site with its new URLs, should this increase be expected? (we have implemented 301 redirects from the old site to the new) 2. What steps would you take to ensure you keep your rankings with the new site? Are they any special tools you recommend? 3. What the best way to increase the ones that have dropped? Page optimization? Link building? Both?
Reporting & Analytics | | pano2 -
Why is this website with worse metrics performing better on serps?
Hi! I would like to ask if anyone has any ideas on why the second website in this analysis is getting position 2 on google and website 1 is getting position 5: http://www.opensiteexplorer.org/comparisons?site=bonobetfair.info&comparisons[0]=www.bonobetfair.com&=Compare The metrics indicate that the first website is clearly superior in (almost) all metrics. Why could this be?
Reporting & Analytics | | inmonova0 -
Strange Visitors To Website
OK, not quite sure how this is happening, but......... I am having referral traffic from online game sites. Actually quite a bit of it and it seems to be raising my bounce rate a bit. B Suggestions anyone? Below is my website: http://www.allianceconcretepumps.com Thank You!!
Reporting & Analytics | | APICDA0