My automated build system is creating a duplicate website
-
Because of the tools my company is using for CI/CD (A CI/CD pipeline helps you automate steps in your software delivery process, such as initiating code builds, running automated tests, and deploying to a staging or production environment.) an extra URL is generated. The canonical for the generated site is that of our main website, but other than that it is the same website.
- Could this new URL compete with our website?
- Will Google count it against us since it is the same content BUT with canonical (it is not noindex-ed)?
- Does it matter?
- Surely others are using this method?
Answers/thoughts will be greatly appreciated. Thank you.
-
Do you have any control over the CI/CD pipeline URL?
If you control the domain enough so that you can be one to have validated and searched console them by all means. But it does not seem like you have the ability to control domain?
my correct?
https://support.google.com/webmasters/answer/7440203?hl=en
If the domain is 3ed party domain then you must trust the third-party or if you control the domain of pages which links or third-party domain URLs are embedded on you can add noindex nofollow
https://www.deepcrawl.com/blog/best-practice/noindex-disallow-nofollow/
I hope that helps,
Tom
-
Unfortunately, since URL is generated from the original site, I cannot change the robots.txt. It uses the same one as the main site. That would exclude adding a noindex meta tag, as well. Any other ideas?
Is there a way to add the duplicate URL to search console & tell google not to crawl?
Thank you.
-
I understand using CI cool
i agree get the bad content being made by CI blocked ASAP
“have an extra URL is generated. The canonical for the generated site is that of our main website, but other than that it is the same website.”
but it’s not the same content being made that will hurt you unless you’re pointing the canonicals to a similar page (get the automated content off your domain)
Remember to add using self pointing canonicals on the good pages you want to be indexed by Google or Search Engines
Hope this is of help,
Tom
-
To answer your questions:
- Technically it could compete with your current site as it's on its own domain, in reality, it's unlikely as you're canonicalizing the pages back to its original and making sure that the content itself through that way is attributed to your original site.
- What I would recommend is excluding the CI/CD site from the engines, through a robots.txt or a similar technique. That way you're making sure that the staging site itself isn't being crawled at all. In the end, I'd say there's very little upside of having that be the case currently.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Hey DA of my website decreased last night what could be the reason?
Hey DA of my website decreased last night what could be the reason?
Reporting & Analytics | | mozmoz90 -
Does intercom support pages and redirect issue can affect the SEO performance of our website?
I noticed that in the redirect issues I have, most of the issues are coming from our Intercom support links. I want to ask, does intercom support pages and redirect issue can affect the SEO performance of our website?
Reporting & Analytics | | Envoke-Marketing0 -
Does Google need Analytics installed to create metrics?
Hi Mozers, Does Google know time on site, number of page visits, bounce rate, etc. even if Google Analytics is NOT installed? Does it measure all that stuff anyway, and if you install the GA code on your site, that is so that YOU can see those metrics? OR can Google only see those metrics if you install GA on your site? Thanks! Jane
Reporting & Analytics | | CalamityJane771 -
Angular website and ranking
Hi guys Unfortunately I have to optimize the angular website, but I don't know how google see my website. Seo quacke (seo extension) doesn't get data from this website: https://cafegardesh.com and sitemap generation tools just crawl 1 page of this website. why? How find that google really crawl and index angular website?
Reporting & Analytics | | denakalami0 -
Links not pointing to my website?
Aloha everyone! I've asked questions to the Moz community before and have always been happy to see how responsive and helpful the community is as a SEO learning new comer! Many thanks in advance for any ideas or insights to solving this problem. So here is my question, I own a wedding photography business here on Kauai and am looking to rank for a few things, Kauai wedding photographer, Kauai Wedding videographer, and Kauai family photographer. That aside I've been pouring more time into getting back links from people I've shot with here on Kauai (local business's websites etc) and have had links dropped in on their sites to, in theory, bump my rankings up from back linking my website. Here's my website http://www.balihaiphoto.com/ Here's a few sites but are not showing up as backlinks/pushing my site higher on SEO results (screen shot as well) http://www.haleleakeiki.org/ (bottom of page footer linking to BaliHaiPhoto.com) and another site here http://www.wishingwellshaveice.com/ with the same thing, footer link pointing to my website. Both sites have been indexed and the shave ice one gets a good amount of traffic, does anyone know why this isn't showing as a back link or if its passing on any link juice my way? Should I do something diffrent here? Let me know what you guys think! Aloha from Kauai, Jon Gibb qsDOM
Reporting & Analytics | | Trey30 -
How can I stop deadlink checker from visiting my website?
It is messing up my Google Analytics traffic reporting. I can't figure out how to get it to stop. Do I filter it out in GA?
Reporting & Analytics | | pmull0 -
Google Analytics Automated Reporting
HI all, I tend to do a big reporting powerpoint deck using screenshots from google analytics and tables I create year end and mid year. It's like an 80 page report for the 10 webisite swe have and then I go ahead and make annotations as I see from the data. That being said this can take a lot of time, up to a 40 hours of time to pull it all together or more which is challenging when you have daily meetings. Anyhow, I've looked into automating and tried a couple things: 1. Tableau- but it keeps crashing and seems tedious 2. Dashlane and supergrabber- seem a bit tedious to set up too. Anyone have ideas on how to better shar ereporting in the organization in this type of format for a website (websites)? Organic, paid, traffic, etc. Laura
Reporting & Analytics | | lauramrobinson322 -
Is there a tool to automatically gather website SEO data?
I am looking for a tool that will crawl a website and create a spreadsheet listing out all key data such as title, meta description, etc. Anyone know of an available tool to do that?
Reporting & Analytics | | jfeitlinger0