Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
Duplicate URL's in Sitemap? Is that a problem?
-
I submitted a sitemap to on Search Console - but noticed that there are duplicate URLs, is that a problem for Google?
-
Hi Luciana! If Logan and/or Matthew answered your question, mind marking one or both of their responses as a "Good Answer" (down in the lower-right of the responses)? It helps us keep track of things, and it gives them a few extra MozPoints.
-
Thank you everyone!
Basically for some reason the system I used to generate the sitemap just has some (not a whole lot) of duplicate URLs, they are exact duplicates. I figured Google would just overlook that.
This was helpful!
Thanks again,
Luciana
-
Generally speaking, this isn't the worst problem you can have with your XML sitemap. In an ideal world, you'll be able to remove duplicate URLs from the sitemap and only submit a single URL for each page. In reality, most larger sites I've encountered have some amount of duplicate content in their XML sitemap with no real major problems.
Duplicate content is really only a major problem if it is "deceptive" in nature. So long as this is just a normal consequence of your CMS, or similar, vs. an attempt to game the rankings you are probably fine. For more about that check out this support article.
The other problem you may encounter is with your search results for those duplicate pages. That article makes mention that Google will pick the URL they think is best (more about that here as well) and the URL they deem the best will be the URL that surfaces in the search results. That may or may not be the same URL you or your visitors would deem best. So, what you might find is Google picked a not great URL (like one with extra parameters) and with the not great URL appearing in the SERPs, your search result isn't as compelling to click on as some other version of the URL may be.
-
Hi,
This isn't necessarily a problem, but XML sitemaps should be as clean as possible before they're uploaded. i.e., no 301'd URLs, no 404s, no dupes, no parameter'd URLs, no canonicalized, etc..
Are they duplicates in the sense that one has caps, and the other doesn't? As in /example.html and /Example.html. If so, you'll want to fix that.
If they're identically formatted URLs, there should be no problem, but you're at duplicate content risk if they're different in anyway and not canonicalized.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
URL Structure on Category Pages
Hi, Currently, we having the following URL Structure o our product pages: All Products Pages: www.viatrading.com/wholesale/283/All_Products.html Category Page: www.viatrading.com/wholesale/4/Clothing.html Product Page: www.viatrading.com/wholesale/product/LOAD-HE-WOM/Assorted-High-End-Women-Clothing-Lots.html?cid=4 Since we are going to use another frontend system, we are thinking about re-working on this URL Structure, using something like this: All Products Pages: www.viatrading.com/wholesale-products/ Category Page: www.viatrading.com/wholesale-products/category/ Product Page: www.viatrading.com/wholesale-products/category/product-title/ I understand this is better for SEO and user experience. However, we already have good traffic on the current URL Structure. Should we use same left-side filters on Category Pages as in All Products Page? Since we are using Faceted Navigation, when users filter the Category (e.g. Clothing) they will see same page as Clothing Category Page. Is that an issue for Duplicate Content? Since we are a wholesale company - I understand is using "/wholesale/products/" in URL for all product pages a good idea? If so, should we avoid word "wholesale" in product-title to avoid repeated word in URL? For us, SKU in URL helps the company employees and maybe some clients identify the link. However, what do you think of using the SEO-friendly product-title, and 301 redirect it to www.viatrading.com/BRTA-LN-DISHRACKS/, so 1st link is only used by company members and Canonicalized 2nd is the only one seen by general public? Thank you,
On-Page Optimization | | viatrading10 -
XML Sitemaps for Property Website
Hi all, I was hoping that someone might have a link to a good example of an XML Sitemap for a large property (real estate) website? Thanks in advance to anyone who does! 🙂 Gavin
On-Page Optimization | | IcanAgency0 -
Should we add our company's name in page title tag or not?
We have been adding our company (Townscript) name in all the page titles. For example, in an event page of Lucknow Conclave: www.townscript.com/lucknowconclave the page title is Lucknow Conclave | Alexis Society | Townscript I read somewhere that it's not necessary to put your company's name in the title tag. Is it right? Please help!
On-Page Optimization | | sanchitmalik0 -
Schema and Rich Snippets What's the difference?
Sorry if this is a daft question but... what is the difference between Rich snippets and Schema markup? Are they one and the same? They seem to be used interchaneably and I'm confused. If someone could give a brief sentence or two about the differences between them that would be great. Thanks
On-Page Optimization | | AL123al1 -
Duplicate Content when Using "visibility classes" in responsive design layouts? - a SEO-Problem?
I have text in the right column of my responsive layout which will show up below the the principal content on small devices. To do this I use visibility classes for DIVs. So I have a DIV with with a unique style text that is visible only on large screen sizes. I copied the same text into another div which shows only up only on small devices while the other div will be hidden in this moment. Technically I have the same text twice on my page. So this might be duplicate content detected as SPAM? I'm concerned because hidden text on page via expand-collapsable textblocks will be read by bots and in my case they will detect it twice?Does anybody have experiences on this issue?bestHolger
On-Page Optimization | | inlinear0 -
Does 'XXX' in Domain get filtered by Google
I have a friend that has xxx in there domain and they are a religious based sex/porn addiction company but they don't show up for the queries that they are optimized against. They have a 12+ year old domain, all good health signs in quality links and press from trusted companies. Google sends them adult traffic, mostly 'trolls' and not the users they are looking for. Has anyone experienced domain word filtering and have a work around or solution? I posted in the Google Webmaster help forums and that community seems a little 'high on their horses' and are trying to hard to be cool. I am not too religious and don't necessarily support the views of the website but just trying to help a friend of a friend with a topic that I have never encountered. here is the url: xxxchurch.com Thanks, Brian
On-Page Optimization | | Add3.com0 -
No follow for html sitemap?
Hi, I have been working on an e-commerce site and have been wondering if i should add the meta robot tag with no follows, on pages like delivery , terms, returns and the html sitemap? From what I have read this seems a good idea, but i am a little confused as what to do with the html sitemap. I can understand that having this on the homepage will allow crawlers quick access to the deeper pages within the site. But would it be better to guide them down the natural route of the category navigation instead? Thanks
On-Page Optimization | | Bmeisterali0 -
Avoiding "Duplicate Page Title" and "Duplicate Page Content" - Best Practices?
We have a website with a searchable database of recipes. You can search the database using an online form with dropdown options for: Course (starter, main, salad, etc)
On-Page Optimization | | smaavie
Cooking Method (fry, bake, boil, steam, etc)
Preparation Time (Under 30 min, 30min to 1 hour, Over 1 hour) Here are some examples of how URLs may look when searching for a recipe: find-a-recipe.php?course=starter
find-a-recipe.php?course=main&preperation-time=30min+to+1+hour
find-a-recipe.php?cooking-method=fry&preperation-time=over+1+hour There is also pagination of search results, so the URL could also have the variable "start", e.g. find-a-recipe.php?course=salad&start=30 There can be any combination of these variables, meaning there are hundreds of possible search results URL variations. This all works well on the site, however it gives multiple "Duplicate Page Title" and "Duplicate Page Content" errors when crawled by SEOmoz. I've seached online and found several possible solutions for this, such as: Setting canonical tag Adding these URL variables to Google Webmasters to tell Google to ignore them Change the Title tag in the head dynamically based on what URL variables are present However I am not sure which of these would be best. As far as I can tell the canonical tag should be used when you have the same page available at two seperate URLs, but this isn't the case here as the search results are always different. Adding these URL variables to Google webmasters won't fix the problem in other search engines, and will presumably continue to get these errors in our SEOmoz crawl reports. Changing the title tag each time can lead to very long title tags, and it doesn't address the problem of duplicate page content. I had hoped there would be a standard solution for problems like this, as I imagine others will have come across this before, but I cannot find the ideal solution. Any help would be much appreciated. Kind Regards5