URL paramters and duplicate content
-
Hello,
I have a 2-fold question:
- Crawl Diagnostics is picking up a lot of Duplicate Page Title errors, and as far as I can tell, all of them are cause by URL parameters trailing the URL.
We use a Magento store, and all filtering attributes, categories, product pages etc are tagged on as URL parameters.
example:
Main URL:
/accessories.htmlDuplicated Title Page URLs:
/accessories.html?dir=asc&order=position
/accessories.html?mode=list
/accessories.html?mode=grid
...and many othersHow can I make the Crawl Diagnostics not identify these as errors?
- Now from an SEO point of view, all these URL parameters are been picked up by google, and are listed in WedMaster Tools -> URL parameters.
All URL parameters are set to "let google decide".
I remember having read that Google was smart enough here to make the right decision, and we shouldn't have to worry about it.
Is this true, or is there a larger issue at hand here?
Thankas!
-
I'm going to look into our canonical tags first - I think this is the best place to start.
Once this is done, I'll wait a week and see if the next scan yields fewer errors.
I'll keep you posted. Thanks.
-
Hi yacpro13. We had a similar problem. We aren't on Magento, but definitely we had the same issue. For us it was a combination of several things. Make sure that your canonical tags are squared away In our case, we had a program running that was automatically writing canonical tags that matched these URLs with the URL parameters attached. Eventually we stopped using the parameters completely, but by then many of these pages had already been indexed (even though they weren't technically separate pages, Google saw them as such). No, Google definitely wasn't smart enough to decide. In our case, Google had sometimes indexed 4-5 versions of the same identical page, and of course we were getting dinged for having duplicate content.
Because we aren't using the parameters any more our solution was to ask Google to remove those URLs from the index. This is a painfully long process for a big site. We've already submitted over 500 remove requests and only about 1/2 have been completed. In your case, since it appears you are currently using parameters, this may not be the best solution. I would start with the canonical tags and investigate those and make sure they aren't part of the problem.
Good luck. Hope this helps a tiny bit.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Duplicate Content in WordPress Taxonomies & Noindex, Follow
Hello Moz Community, We are seeing duplicate content issues in our Moz report for our WordPress site’s Tag pages. After a bit of research, it appears one of the best solutions is to set Tag pages to “no index, follow” within Yoast. That makes sense, but we have a few questions: In doing this, how are we affecting our opportunity to show up in search results? Are there any other repercussions to making this change? What would it take to make the content on these pages be seen as unique?
Moz Pro | | CoreyHicks1 -
Duplicate content flagged by Moz that's not actually duplicate content at all
Hi, Moz has flagged a whole lot of pages as dupe content, but I cannot see how they qualify as such.
Moz Pro | | Caro-O
Not sure if I'm allowed to post actual URLs here....happy to if I can, but I feel certain that the pages are not 90% similar. Has anyone else had this experience? ~Caro1 -
Excel tips or tricks for duplicate content madness?
Dearest SEO Friends, I'm working on a site that has over 2,400 instances of duplicate content (yikes!). I'm hoping somebody could offer some excel tips or tricks to managing my SEOMoz crawl diagnostics summary data file in a meaningful way, because right now this spreadsheet is not really helpful. Here's a hypothetical situation to describe why: Say we had three columns of duplicate content. The data is displayed thusly: | Column A | Column B | Column C URL A | URL B | URL C | In a perfect world, this is easy to understand. I want URL A to be the canonical. But unfortunately, the way my spreadsheet is populated, this ends up happening: | Column A | Column B | Column C URL A | URL B | URL C URL B | URL A | URL C URL C | URL A | URL B | Essentially all of these URLs would end up being called a canonical, thus rendering the effect of the tag ineffective. On a site with small errors, this has never been a problem, because I can just spot check my steps. But the site I'm working on has thousands of instances, making it really hard to identify or even scale these patterns accurately. This is particularly problematic as some of these URLs are identified as duplicates 50+ times! So my spreadsheet has well over 100K cells!!! Madness!!! Obviously, I can't go through manually. It would take me years to ensure the accuracy, and I'm assuming that's not really a scalable goal. Here's what I would love, but I'm not getting my hopes up. Does anyone know of a formulaic way that Excel could identify row matches and think - "oh! these are all the same rows of data, just mismatched. I'll kill off duplicate rows, so only one truly unique row of data exists for this particular set" ? Or some other work around that could help me with my duplicate content madness? Much appreciated, you Excel Gurus you!
Moz Pro | | FMLLC0 -
Duplicate Page Titles and Content
The SeoMoz crawler has found many pages like this on my site with /?Letter=Letter, e.g. http://www.johnsearles.com/metal-art-tiles/?D=A. I believe it is finding multiple caches of a page and identifying them as duplicates. Is there any way to screen out these multiple cache results?
Moz Pro | | johnsearles0 -
Help with duplicate title tags?
I was looking in Google webmaster tools and it says I have 95 duplicate title tags for my site Noah's Dad. When I look through the list it appears the pages with duplicate title tags are some of my category pages, archive pages, and some author pages... Not sure if you guys can use some of the tools to see what is actually showing up duplicate or not, and if you need more info just let me know. But I wanted to see if this is something I should be concerned with? Should WMT also say 0 in duplicate content? It seems like when I started my blog I was told no to be conceded with this sort of stuff in gwmt. Anyways...I just wanted to see what you guys think. (By the way, is there any way to tell what this duplicate content is having (or has had) on my SERP results? Thanks.
Moz Pro | | NoahsDad0 -
Duplicate Content Issues with WordPress
I'm having some difficulty with a few of the sites I'm managing right now. When I run a report here, I'm getting a duplicate content issue with sites that I'm running through WordPress. Sites running on a different CMS are not getting the issue. The duplicate content is being listed as from two URL's that are identical. I checked trailing slash, spelling, capitalization, everything. It looks like the same site is being marked as two with duplicate content. Does anyone have any ideas of what could be causing this and/or what I may be able to do to resolve the issue (or if it's really something to worry about or not)? Thanks. (and thanks for helping the new guy!)
Moz Pro | | DeliaAssociates0 -
Should I worry about duplicate content errors caused by backslashes?
Frequently we get red-flagged for duplicate content in the MozPro Crawl Diagnostics for URLs with and without a backslash at the end. For example: www.example.com/ gets flagged as being a duplicate of www.example.com I assume that we could rel=canonical this, if needed, but our assumption has been that Google is clever enough to discount this as a genuine crawl error. Can anyone confirm or deny that? Thanks.
Moz Pro | | MackenzieFogelson0 -
Looking for a tool that can pull OSE stats for a bulk amount of URLs
I know that people have developed inhouse tools with the OSE API that can analyze thousands of URLs and pull metrics like PA, inbound links, etc. I need to analyze about 80k URLs and sort them by authority and I was hoping that someone could point me to a tool that can do this or let me use their tool. I'm willing to pay for access to it. We could build it inhouse, I imagine that it would be pretty easy, but our IT resources are stretched too thin right now.
Moz Pro | | Business.com0