My automated build system is creating a duplicate website
-
Because of the tools my company is using for CI/CD (A CI/CD pipeline helps you automate steps in your software delivery process, such as initiating code builds, running automated tests, and deploying to a staging or production environment.) an extra URL is generated. The canonical for the generated site is that of our main website, but other than that it is the same website.
- Could this new URL compete with our website?
- Will Google count it against us since it is the same content BUT with canonical (it is not noindex-ed)?
- Does it matter?
- Surely others are using this method?
Answers/thoughts will be greatly appreciated. Thank you.
-
Do you have any control over the CI/CD pipeline URL?
If you control the domain enough so that you can be one to have validated and searched console them by all means. But it does not seem like you have the ability to control domain?
my correct?
https://support.google.com/webmasters/answer/7440203?hl=en
If the domain is 3ed party domain then you must trust the third-party or if you control the domain of pages which links or third-party domain URLs are embedded on you can add noindex nofollow
https://www.deepcrawl.com/blog/best-practice/noindex-disallow-nofollow/
I hope that helps,
Tom
-
Unfortunately, since URL is generated from the original site, I cannot change the robots.txt. It uses the same one as the main site. That would exclude adding a noindex meta tag, as well. Any other ideas?
Is there a way to add the duplicate URL to search console & tell google not to crawl?
Thank you.
-
I understand using CI cool
i agree get the bad content being made by CI blocked ASAP
“have an extra URL is generated. The canonical for the generated site is that of our main website, but other than that it is the same website.”
but it’s not the same content being made that will hurt you unless you’re pointing the canonicals to a similar page (get the automated content off your domain)
Remember to add using self pointing canonicals on the good pages you want to be indexed by Google or Search Engines
Hope this is of help,
Tom
-
To answer your questions:
- Technically it could compete with your current site as it's on its own domain, in reality, it's unlikely as you're canonicalizing the pages back to its original and making sure that the content itself through that way is attributed to your original site.
- What I would recommend is excluding the CI/CD site from the engines, through a robots.txt or a similar technique. That way you're making sure that the staging site itself isn't being crawled at all. In the end, I'd say there's very little upside of having that be the case currently.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
My website has been declining in rankings!
Hello everyone, I have been doing the SEO on the following site: www.painters-decorators-london.co.uk for the past few years and the rankings are slowly declining since last year. (not the recent Google update). I can't seem to figure out what exactly is causing this to happen The page authority and domain authority is higher than my competitors. The on page SEO I believe is done appropriately but I'm not sure whats causing this decline. How can I find out exactly using the MOZ Pro account that I have. I don't think I'm totally aware of the important metrics that one can use on this site to determine the problem. Please advise!
Reporting & Analytics | | Ready2Paint0 -
Angular website and ranking
Hi guys Unfortunately I have to optimize the angular website, but I don't know how google see my website. Seo quacke (seo extension) doesn't get data from this website: https://cafegardesh.com and sitemap generation tools just crawl 1 page of this website. why? How find that google really crawl and index angular website?
Reporting & Analytics | | denakalami0 -
Is there an automated way to determine which pages of your website are getting 0 traffic?
I'm doing a content audit on my company website and want to identify pages with zero traffic. I can use GA for low traffic, but not zero traffic. I can do this manually, but it would take a long time. Are there any tools to help me determine these pages?
Reporting & Analytics | | Ksink0 -
Creating Meta Tags for a Web Hosting company
My client has recently had his site rewritten by a company which supplies a website/hosting/seo service. The client still wants to use me for establishing his site locally and for SEM. My problem is that the code is not accessible to me. They have set up GA tracking which feeds into the reports dashboard. There is no way of actually access the raw data. When I asked for access to the GA account I got this message from bOline solutions: "Your SEO consultant will need to create a new GA account and then add the meta tag to the reports tab of your site" Because I am still very much learning GA I do not know what meta tag they are referring too. I'm therefore stuck as to how to create it. I've started learning about "Google Tag Manager" and am working through "Digital Analytics Fundementals", I've a feeling this is just a question of terminology - can anyone more experience in the subject help me?
Reporting & Analytics | | catherine-2793880 -
How Google measure website bounce rate ?
Bounce rate is a SEO signal, but how Google measures it ? There is any explanation about this ? Does Google uses Analytics ? Maybe time between 2 clics in search results ? Thanks
Reporting & Analytics | | Max840 -
The client's website serves as the main referral?
Hi mozzers, I have this weird case where one of my client's first referral is its own website!! I am really confused especially that I have checked there www vs non www and the non www is redirected to the www. This means that it resolve to one version which is good! Any thoughts on why the main referral is its own site? Thanks
Reporting & Analytics | | Ideas-Money-Art0 -
Dramatic Increase in referrals from own website
The past few weeks I've been wracking my brain to figure out why on earth my branded searches could be dropping off at 30-40%. Well today, I realize I've had a dramatic increase in referrals from my own site (the non www verison). I'm talking 150 in March of last year to 5k in March of this year drastic. My 301 redirects haven't changed as far as I know -- I've had them set to redirect from the non www. to the www. for at least a year or two. I'm assuming visitors from search engines are somehow getting the non www version and the redirect is attributing the traffic to referrals instead of search. The drop in search traffic and the increase in referral traffic fall on the same day. Does that sound right/possible? If so, how do I fix this? My traffic stats for two clients are all screwy because of this. I want to make sure whatever solution I implement won't hurt my search traffic numbers any more 🙂 Has anyone else seen this happen recently? I could imagine an anomaly with one site but I find it odd that it could be two client sites. I still have some others to check. Thanks in advance! Leslie
Reporting & Analytics | | LeslieVS0 -
Distribution of SEOMOZ Ranking Metrics across all Monitored Websites
Hi, I'm looking into building a tool which would incorporate SEOMOZ ranking metrics and I have a few questions with regard to the ranking data which will help me develop it correctly. I have done a few searches, for the information but haven't found an answer to these questions on seomoz - if they exist somewhere then I would be grateful if you could point me in the right direction. 1/. When looking at the values of metrics such as MozRank and DA, is there any information on how this values are distributed amongst the ranked website - EG The mean / median values of DA and MozRank etc of all websites monitored is X & Y . If there were any distribution graphs for these values then even better. 2/. Along the lines of the above - are there any online resources showing similar information for google PR? 3/. What is the calcuation used in the scale of MozRank and DA - ie We have a scale of 1-10 for these values - be we also know the scale in not linear but logrithmic - ie Double the Links of a DA 5 website and you would not end up with a DA 10 website - so what is the calculation which defines the scale. A WBF about google PR by Rand indicated that a value of between 8-9 is used in Google PR - but what about the values used for DA and MozRank etc? Many thanks for your help
Reporting & Analytics | | James770