Duplicate Sub-domains Being Indexed
-
Hi all,
I have this site that has a sub-domain that is meant to be a "support" for clients. Some sort of FAQ pages, if you will. A lot of them are dynamic URLs, hence, the title and most of the content are duplicated.
Crawl Diagnostics found 52 duplicate content, 138 duplicate title and a lot other errors.
My question is, what would be the best practice to fix this issue? Should I noindex and nofollow all of its subdomains? Thanks in advance.
-
Thanks a lot! I will definitely try that.
-
If you don't want these indexed, first put a noindex tag on all pages. Leave the follow alone as the engine still needs to find the pages to change the index status.
Add the domain to GWMT then request a removal all the pages.
Allow this to take effect then add a robots disallow to the entire sub-domain.
Your domain then be cleaned from the index and the duplication won't be an issue.
-
The site has a few subdomains. For example (below examples are not my site):
- https://support.medialayer.com/index.php?_m=knowledgebase&_a=view
- https://support.medialayer.com/index.php?_m=knowledgebase&_a=viewarticle&kbarticleid=9&nav=0
So basically, the pages got indexed, with the same title, and partial/whole duplicate content.
So, by simply making some changes on the robots.txt will fix this issue? Thanks.
-
How are all of these subdomains being created? Do you have just one subdomain you are worried about or multiple subdomains?
If you can do a robots.txt disallow on certain duplicate folders/files that you don't want indexed, that might help.
Let me know some more info first.
Scott.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Duplicate content across different domains in different countries?
Hi Guys, We have a 4 sites One in NZ, UK, Canada and Australia. All geo-targeting their respective countries in Google Search Console. The sites are identical. We recently added the same content to all 4 sites. Will this cause duplicate content issues or any issues even though they are in different countries and geo-targeting is set? Cheers.
Intermediate & Advanced SEO | | wickstar0 -
Our website is not being indexed
We have an issue with a site that we can't get to the bottom of. This site: (URL removed) is not being properly indexed. When we do a search for (URL removed) in google.com.au. The site appears as the 4th listing with the following title and description: (Title removed) A description for this result is not available because of this site's robots.txt – learn more. We have checked the site's robots.txt and can see its been now implemented correctly: (URL removed) About a week ago, we also went into Webmaster Tools and submitted a request for Google to recrawl our site. We are unsure what the issue is that is causing the site to not be properly indexed and how to resolve it. Any assistance on this topic would be most appreciated!
Intermediate & Advanced SEO | | Gavo0 -
Do links to a domain that re-directs to my domain pass link equity?
Hi guys. We've recently taken control of a third-party site and we're going to set up a domain re-direct so any traffic comes to our site. With any existing links that the third-party site has, will these pass link equity to our main site through the redirect? Thanks, Paul
Intermediate & Advanced SEO | | kevinliao0 -
Remove content that is indexed?
Hi guys, I want to delete a entire folder with content indexed, how i can explain to google that content no longer exists?
Intermediate & Advanced SEO | | Valarlf0 -
Website Consolidation To Sub Domains or Leave Stand Alone
For a real estate SEO client they have their corporate site and then for each of their communities (10 of them) each community has their own website domains. One of their team members met with another search agency who recommended they move (consolidate) all their community domains under the corporate site as a sub domain. For example let's say their main site was www.maincompany.com and one of their communities was www.localcompany.com the other firm recommended they move that existing site to become localcompany.maincompany.com and for the other 9 communities to do the exact same thing. They shared that it would really help the corporate site and each of the communities improve search rankings. I am struggling to see how this could be possible and was hoping to get some perspective as the client has asked me to come in and give my opinion if they should proceed with this consolidation. Google has indexed each of their community sites and each site gets a decent amount of search traffic and rankings. Due to that I can't see any benefit to doing this. Since each sub domain would be considered a different site than it essentially is what they already have today so it does not raise domain authority for the main company site. Since, each community has a very different brand there would be little reason to go the main company site. What I mean by that is if a user went to localcompany.maincompany.com site I thought that some may (at least out of curiosity) remove the sub-domain to see the maincompany.com site. The look and feel of each are so different it would potentially cause user confusion too. So my thoughts are this would be a negative for both the search engines and user. So I can share factual pros/cons with clients, do you have any thoughts to the pro's / con's of this approach to consolidate/move other existing sites under a sub domain of the main corporate site?
Intermediate & Advanced SEO | | jfeitlinger1 -
Indexing issue?
Hey guys when I do a search of site:thetechblock.com query in Google I don't seem to see any recent posts (nothing for August). In Google webmaster I see that the site is being crawled (I think), but I'm not sure. I also see the the sitemaps are being indexed but again it just seems really odd that I'm not seeing these in Google results. SEO seems all good too with SEO Moz. Is there something I'm not getting?
Intermediate & Advanced SEO | | ttb0 -
Ranking with other pages not index
The site ranks on page 4-5 with other page like privacy, about us, term pages. I encounter this problem allot in the last weeks; this usually occurs after the page sits 1-2 months on page 1 for the terms. I'm thinking of to much use the same anchor as a primary issue. The sites in questions are 1-5 pages microniche sites. Any suggestions is appreciated. Thank You
Intermediate & Advanced SEO | | m3fan0 -
How do I fix the error duplicate page content and duplicate page title?
On my site www.millsheating.co.uk I have the error message as per the question title. The conflict is coming from these two pages which are effectively the same page: www.millsheating.co.uk www.millsheating.co.uk/index I have added a htaccess file to the root folder as I thought (hoped) it would fix the problem but I doesn't appear to have done so. this is the content of the htaccess file: Options +FollowSymLinks RewriteEngine On RewriteCond %{HTTP_HOST} ^millsheating.co.uk RewriteRule (.*) http://www.millsheating.co.uk/$1 [R=301,L] RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /index\.html\ HTTP/ RewriteRule ^index\.html$ http://www.millsheating.co.uk/ [R=301,L] AddType x-mapp-php5 .php
Intermediate & Advanced SEO | | JasonHegarty0