Duplicate page report
-
We ran a CSV spreadsheet of our crawl diagnostics related to duplicate URLS' after waiting 5 days with no response to how Rogerbot can be made to filter.
My IT lead tells me he thinks the label on the spreadsheet is showing “duplicate URLs”, and that is – literally – what the spreadsheet is showing.
It thinks that a database ID number is the only valid part of a URL. To replicate: Just filter the spreadsheet for any number that you see on the page. For example, filtering for 1793 gives us the following result:
|
URL
http://truthbook.com/faq/dsp_viewFAQ.cfm?faqID=1793
http://truthbook.com/index.cfm?linkID=1793
http://truthbook.com/index.cfm?linkID=1793&pf=true
http://www.truthbook.com/blogs/dsp_viewBlogEntry.cfm?blogentryID=1793
http://www.truthbook.com/index.cfm?linkID=1793
|
There are a couple of problems with the above:
1. It gives the www result, as well as the non-www result.
2. It is seeing the print version as a duplicate (&pf=true) but these are blocked from Google via the noindex header tag.
3. It thinks that different sections of the website with the same ID number the same thing (faq / blogs / pages)
In short: this particular report tell us nothing at all.
I am trying to get a perspective from someone at SEOMoz to determine if he is reading the result correctly or there is something he is missing?
Please help. Jim
-
Hi Jim!
Thanks for the question. One thing we should clarify before we move forward is that the Pro app doesn't actually report on duplicate URLs, but we do report when we find duplicate title tags or content.
Duplicate titles just refer to when we find the same title tag on more than one page. In one example from your diagnostics, we're reporting the title tag 'Truthbook Religious News' is being used in multiple pages (http://screencast.com/t/GYCKNfAoj).
Duplicate content is content we see on the source code of your pages that is identical or nearly identical and would cause the pages to compete against each other for rankings. To fix either of these you have a several options:
- Set up a 301 redirect to have the pages you would consider duplicate redirect to the main page.
- Change the content/title tags enough that they won't be considered duplicates - Canonicalize the content you would consider duplicates.
Most developers will go for the latter two options so that the pages will still be reachable by visitors. You can find out more about how to implement these in our Help Hub.
To answer your other questions:
1 - At the time of the crawl, we were able to get to sub domain pages from other pages on your site. The sub domains were also resolving separately, but they seem to be redirecting to your root domain now, so your next crawl should reflect this.
2 - Running a curl for the print versions of your pages, I see "no follow" tags related to Wikipedia links embedded (http://screencast.com/t/reYjeLLPvWG3) in the doc, but I'm not finding any "no index tags" (http://screencast.com/t/DsXMZInngSzH). This would be why you're seeing us crawling those pages.
3 - As I mentioned above, our crawler looks for similarities in the source code of pages when reporting on duplicate content. Since no one knows exactly how similar content would need to be for the search engines to consider it a duplicate, we err on the side of caution and recommended best practices when reporting them. Using one of the methods mentioned above and detailed in our Help Hub should resolve this for you
Let me know if you have any other questions!
Best,
Sam
Moz Helpster - Set up a 301 redirect to have the pages you would consider duplicate redirect to the main page.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Duplicate content : what are best solutions
Hello i am a beginner in website, I got my first report the report saying that there are some duplicate content. I would like to know What can I do to solve that issue ?
Moz Pro | | Dieumerci0 -
Seomoz legacy pages?
Hello, I am finding that I miss several of the old seomoz sections. The legacy tools in particular like the visual website comparison. Where is that now? Also, where is the ongoing list of the top 100 sites? So much was lost in the shift to MOZ, I hope some of the good old stuff is still available. Thank you, Nolan
Moz Pro | | QuietProgress0 -
OSE - advanced reports.
Trying to run advanced reports, but am being told I've hit my daily report limited - and points me to signing up for a Pro membership - although I already have one. Been like this the last couple of days. Anyone else experiencing this? Thanks.
Moz Pro | | newstd1000 -
Is there any report / tool that gives me last cache date for each page on my site ?
my site has several hundred pages, and it is important for me to know last crawl date of each page as well as number of pages cralwed in a particualr period ( from / to date ). is there any report in seomoz that can help for this ? or any other suggestion ?
Moz Pro | | elegantmicroweb0 -
Page Authority Bulk Check?
Hi all, Was wondering if maybe using the SEOmoz API there was a way to check in excel the page authority of 500 URLs or more? Or is there a different way to do this? Thanks, Carlos
Moz Pro | | Carlos-R0 -
How Do I deal with duplicate page titles for pages on eCommerce site
Hi We have an ecommerce site selling physical products. There are a few areas where the products run into two pages. I have used canonical meta tags and next and prev meta tags too. Despite this SEOMOZ reports are still displaying these as warnings for duplicate page titles. An example would be /brand_name/range/ <link rel="next" href="/brand_name/range/?page=2" /> <link rel="<a class="attribute-value">canonical</a>" href="/band_name/range/"/> /brand_name/range/?page=2 <link rel="<a class="attribute-value">prev</a>" href="/brand_name/range/" /> <link rel="<a class="attribute-value">canonical</a>" href="/band_name/range/?page=2"/> Should I be doing something different?
Moz Pro | | wouldBseoKING0 -
Archived campaign and automatic reports
Hi I set up the standard reports under Reports new and am still getting them emailed with no data. Just want to stop receiving as I have archived the campaign Thanks
Moz Pro | | Alexanders0 -
Personalised PRO Dashboard reports?
Hi guys, I'm loving the SEOMoz PRO dashboard and have learnt so much from the campaigns I've been running. I was wondering if I could personalise the reports to have my logo on them instead of SEO MOz's? At the moment, I export them and try recreate them offline, but it wouold be way easier if I could just send them the reports PDF'ed up straight from the dashboard.
Moz Pro | | Lindsay_D0