Duplicate URLs
-
A campaign that I ran said that my client's site had some 47,000+ duplicate pages and titles. I was wondering how I can possibly set that many 301 redirects, but a Moz help engineer said it has a lot to do with session IDs. See this set of duplicate URLs:
http://www.lumberliquidators.com/ll/c/engineered-hardwood-flooring (clearly the main URL for the page)
http://www.lumberliquidators.com/ll/c/engineered-hardwood-flooring?PIPELINE_SESSION_ID=0ac00a2e0ad53eb90cb0b0304d178fc1
http://www.lumberliquidators.com/ll/c/engineered-hardwood-flooring?PIPELINE_SESSION_ID=0ac3039d0ad4af2720b3ccd2238547ab
http://www.lumberliquidators.com/ll/c/engineered-hardwood-flooring?PIPELINE_SESSION_ID=0ac071ed0ad4af292684b0746931158fTo a crawler, that looks like 4 different pages, when it's clear that they're actually all different URLs for the same page. I was wondering if some of you, maybe with experience in site architecture, would have insight into how to address this issue?
Thanks
Alan
-
Hm, have you looked into rel canonical?
If those are all stand alone pages, you will have to redirect, if they are no longer active, or if they can be replaced by the original page.
Andy is correct, those pages likely are not 'created' with intent.
You should look at what is causing this issue and start there. If not, you are going to be redirecting till the cows come home.
If you are deciding on going through 301's, you may want to take a step back and look at the folders of the entire domain. /ll/ is a folder but not a page, nor is /ll/c/.
Good Luck, Alan!
-
A quick way would be to disallow crawling of all pages starting with /?PIPELINE. That will prevent Google from seeing them. You can do this by adding the following into your robots.txt file...
Disallow: /*?PIPELINE
However, you want to get to the root cause, which will be something to do with the system generating these. Ideally, this needs to be fixed.
-Andy
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Canonical URLs all show trailing slash on main site pages - using Yoast SEO for Wordpress - how to correct
We are using Yoast for a number of our sites. We use naked domain as the canonical. I have noticed in the header tags that all our sites show the canonical URLs as having a trailing slash: Example: http;//foxspizzajc.com, when I look at the source code, it shows the canonical as http;//foxspizzajc.com/ Of course, it is much more likely that all sites that link to us will not use the trailing slash - so preferably we do not want that to be the canonical - among other reasons. Does this need to be fixed so the trailing slash is removed? I cannot see how to do this in Yoast SEO or in Permalinks structure for Wordpress. Sorry for my ignorance. Thanks for any help.
Moz Pro | | Adam_RushHour_Marketing1 -
Why is OSE showing no data for this URL?
Hi all, Does anyone have any ideas as to why OSE might not have any data for this URL: http://www.ccisolutions.com/StoreFront/product/shure-slx24-sm58-wireless-microphone-system-j3 It is not a new page at all. It's been on the site for years. Is OSE being quirky? Or is there an underlying problem with this page? Thanks in advance for any light you can shed on this, Dana
Moz Pro | | danatanseo0 -
Adding canonical still returns duplicate pages
According to SEOmoz, several of my campaigns show that I have duplicate pages (SEOmoz Errors). Upon reading more about how to resolve the issue, I followed SEOmoz's suggestion to add rel='canonical' <links>to each page. After the next SEOmoz crawl, the number of SEOmoz Errors related to duplicate pages remained the same and the number of SEOmoz notices shot up indicating that it recognized that I added rel='canonical'.</links> I'm still puzzled as to why the SEOmoz errors did not go down with respect to duplicate page errors after I added rel='canonical', especially since SEOmoz noticed that I added them. Can anyone explain this to me? Thanks,
Moz Pro | | MOZ2
Scott.0 -
Crawl Diagnostics returning duplicate content based on session id
I'm just starting to dig into crawl diagnostics and it is returning quite a few errors. Primarily, the crawl is indicating duplicate content (page titles, meta tags, etc), because of a session id in the URL. I have set-up a URL parameter in Google Webmaster Tools to help Google recognize the existence of this session id. Is there any way to tell the SEOMoz spider the same thing? I'd like to get rid of these errors since I've already handled them for the most part.
Moz Pro | | csingsaas0 -
Duplicate page content showing up with proper use of canonical tag
Hi, In the Crawl diagnostics reports, I'm getting lots of duplicate errors warnings e.g. duplicate page title. In most cases these are tracking urls and the page has a canonical tag pointing to the original page. It would be helpful if the crawl analysis reports could separate these out from ones that are of genuine concern. It can also happen when there's a noindex tag on a page. Thanks, Leigh
Moz Pro | | Leighm0 -
Crawl reports urls with duplicate content but its not the case
Hi guys!
Moz Pro | | MakMour
Some hours ago I received my crawl report. I noticed several records with urls with duplicate content so I went to open those urls one by one.
Not one of those urls were really with duplicate content but I have a concern because website is about product showcase and many articles are just images with href behind them. Many of those articles are using the same images so maybe thats why the seomoz crawler duplicate content flag is raised. I wonder if Google has problem with that too. See for yourself how it looks like: http://by.vg/NJ97y
http://by.vg/BQypE Those two url's are flagged as duplicates...please mind the language(Greek) and try to focus on the urls and content. ps: my example is simplified just for the purpose of my question. <colgroup><col width="3436"></colgroup>
| URLs with Duplicate Page Content (up to 5) |0 -
How can I Pull OSE Data for Multiple URL's at once
I'm putting together a link prospecting csv (very basic/simple). I'm doing my own manual hunting for link prospects and compiling them in a list in that excel doc. Once I'm done with that, I want to pull OSE data on a larger scale (MozRank, PA, DA, etc.). I know Niel Bosma's SEO tools for Excel exists, but I have a Mac, and it's not available for that. And I can't really pay for any of the big tools right now (ie BuzzStream). Does anybody know of a good tool or way of going about pulling this data in a way that will save time? As opposed to pulling data for each URL one by one. ANY tips would be GREATLY appreciated.
Moz Pro | | MichaelWeisbaum0 -
Company Name in Page Title creating thousands of "Duplicate Page Title" errors
I am new, and I just got back my crawl results (after a week or more). The first thing I noticed is that the "duplicate page title" is in the thousands, my urls and page titles are different. The only thing I can see is that our company name is at appended to the name of every title. I did search and found one other person with this problem, but no answer was given. Can anyone offer some advice? This doesn't seem right... Thanks,
Moz Pro | | AoyamaJPN0