Perplexed by last MOZ crawling duplicate content errors
-
In the last crawler issues report from MOZ I can see many many pages listed as duplicate content with 0 duplicate urls.
Like this: http://imgur.com/fbikRVq
I am puzzled, what does it mean?
-
Even in the last crawl report the bug is still there, any idea when it will be fixed?
-
Thanks for the answer/update.
-
Hi Max and Doug - this is currently being recognized as a bug and we have it currently being worked on as we speak. Sorry for the confusion in the short term!
-
We have not had that happen. All the "Duplicate Content" items list a number of duplicates that's > 0.
It is possible that Moz is buggy.
-
But was your report showing the list of url considered duplicate of that url?
In my case on the right I have the duplicate url, but on the left (where the list of other url the crawler consider duplicate should be) there's written 0 page duplicate, and the list is empty.
I am aware even if two pages looks different to me could be considered duplicate by rogerbot, but in the past it was always showing the number of duplicate pages found and the list of duplicate url.
-
We had the same concern and asked MOZ support this question: Why are we getting duplicate content warnings for pages that are clearly different?
We received the response below. Our takeaway is that we will continue to take these warning into consideration, but apply our own expertise to determine if action is needed.
Response from support: Thanks for reaching out, and sorry for the confusion! Duplicate content is always kind of a tricky issue. While you or I can qualitatively determine that there are differences between these pages, crawlers are dependent on more quantitative means to determine duplicate content. When they view the pages, one part of the process is to examine the similarity of the pages' code and look for close matches to determine duplicates; this appears to be the issue here. I stuck these URLs into a similar page checker (http://www.webconfs.com/similar-page-checker.php), and it indicated that there was quite a high degree of similarity.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Crawl Test is now On-Demand Crawl!
If you've been with Moz a while, you may have used our old Crawl Test tool. A year ago we launched an all new, campaign-based Site Crawl (with an entirely rebuild crawl engine), but Crawl Test fell into disrepair and we haven't had a solid tool for crawling non-campaign domains. I'm happy to announce that we've just launched an all new On-Demand Crawl, built on the new Site Crawl engine, with a UI that's focused on quick insights. Moz Pro Standard tier customers can run up to 5 crawls per month at 3,000 page per crawl (crawls are saved for 90 days), with per-month limits increasing at higher levels. Most On-Demand Crawls should run in a few minutes, making the tool perfect to get quick insights for sales meetings, vetting prospects, or analyzing competitors. We've written up a sample case study or logged-in customers can go directly to On-Demand Crawl. Try it out -- we'd love to hear your use cases (either here or in the blog post comments).
Moz Bar | | Dr-Pete6 -
When I try to run a Moz report, it sends me to a 404 page?
Hey there. I'm trying to export a .pdf to send to my client. When I click "export pdf", the page sits for a second then goes to a 404 page? I've never seen this before. Is anyone else getting this problem?
Moz Bar | | TaylorRHawkins2 -
Error in Duplicate Content Being Reported - Pages Aren't Actually Duplicates
The recent crawl of one of our sites revealed a high number of duplicate content issues. However, when I viewed the report for pages with duplicate content I noticed almost all of them are not duplicates. For example, these two pages are marked as dupes:
Moz Bar | | M_D_Golden_Peak
https://www.writersstore.com/publishers/hollywood-creative-directory
https://www.writersstore.com/authors/g-miki-hayden These are thin as far as content goes but definitely not duplicates. Any recommendations or ways to adjust the settings so that these false positives aren't clogging up our site crawl report?0 -
How do I go about fixing my High Priority issues that SEO moz says I have on a PHP site?
I am been trying to deal with this problem for some time now. I have talked to several IT people and SEO moz. None seem to know how to fix these issues on the type of site our company is. Our biggest issue with is Duplicate Page Content. We also have some title issues. Our site is built with PHP coding and variable, meaning the site is not a typical static website. We have a handful of pages that are dynamic depending on what the users chooses to see and do. So, my problem is I can't just go to a specific page and put the canonical or the redirect. It isn't multiple pages for our category pages, for example, it is just one that builds the page depending on the search. Please help!
Moz Bar | | JoshMaxAmps0 -
Moz Crawler URL paramaters & duplicate content
Hi all, this is my first post on Moz Q&A 🙂 Questions: Does the Moz Crawler take into account rel="canonical" for search results pages with sorting / filtering URL parameters? How much time does it take for an issue to disappear from the issues list after it's been corrected? Does it come op in the next weekly report? I'm asking because the crawler is reporting 50k+ pages crawled, when in reality, this number should be closer to 1000. All pages with query parameters have the correct canonical tag pointing to the root URL, so I'm wondering whether I need to noindex the other pages for the crawler to report correct data?: Original (canonical URL): DOMAIN.COM/charters/search/mx/BS?search_location=cabo-san-lucas Filter active URL: DOMAIN.COM/charters/search/mx/BS?search_location=cabo-san-lucas&booking_date=&booking_days=1&booking_persons=1&priceFilter%5B%5D=0%2C500&includedPriceFilter%5B%5D=drinks-soft Also, if noindex is the only solution, will it impact the ranking of the pages involved? Note: Google and Bing are semi-successful in reporting index page count, each reporting around 2.5k result pages when using the site:DOMAIN.com query. The rel canonical tag was missing for a short period of time about 4 weeks ago, but since fixing the issue these pages still haven't been deindexed. Appreciate any suggestions regarding Moz Crawler & Google / Bing index count!
Moz Bar | | Vukan_Simic0 -
Checking Do-follow using Moz Bar
Does anyone know how to check if a link is a do-follow or no-follow using the Moz bar? I believe there's a function that highlights and color codes the link so as to tell? Can't seem to turn this function on in my Moz Bar settings. Am I missing something completely obvious?
Moz Bar | | Gavo0 -
Site crawl errors - download list of all urls
Hi Ive provided my clients developers with the pdf reports of crawl errors but these seem to miss some urls I see there are lots of csv file download/email options Will the email csv button send a report of everything listing all urls that are missing from the pdfs ? if not will the more specific csv reports Would be good if i can press 1 button and get all issues listed with all urls It does look like this happens but i just want confirmed best way asap since need to provide reports urgently, any guidance much appreciated ? All Best Dan
Moz Bar | | Dan-Lawrence0 -
Chrome moz toolbar page analysis not loading
Often the chrome moz toolbar page analysis doesn't load just says Please wait until the page finishes loading.
Moz Bar | | genkee0