Perplexed by last MOZ crawling duplicate content errors
-
In the last crawler issues report from MOZ I can see many many pages listed as duplicate content with 0 duplicate urls.
Like this: http://imgur.com/fbikRVq
I am puzzled, what does it mean?
-
Even in the last crawl report the bug is still there, any idea when it will be fixed?
-
Thanks for the answer/update.
-
Hi Max and Doug - this is currently being recognized as a bug and we have it currently being worked on as we speak. Sorry for the confusion in the short term!
-
We have not had that happen. All the "Duplicate Content" items list a number of duplicates that's > 0.
It is possible that Moz is buggy.
-
But was your report showing the list of url considered duplicate of that url?
In my case on the right I have the duplicate url, but on the left (where the list of other url the crawler consider duplicate should be) there's written 0 page duplicate, and the list is empty.
I am aware even if two pages looks different to me could be considered duplicate by rogerbot, but in the past it was always showing the number of duplicate pages found and the list of duplicate url.
-
We had the same concern and asked MOZ support this question: Why are we getting duplicate content warnings for pages that are clearly different?
We received the response below. Our takeaway is that we will continue to take these warning into consideration, but apply our own expertise to determine if action is needed.
Response from support: Thanks for reaching out, and sorry for the confusion! Duplicate content is always kind of a tricky issue. While you or I can qualitatively determine that there are differences between these pages, crawlers are dependent on more quantitative means to determine duplicate content. When they view the pages, one part of the process is to examine the similarity of the pages' code and look for close matches to determine duplicates; this appears to be the issue here. I stuck these URLs into a similar page checker (http://www.webconfs.com/similar-page-checker.php), and it indicated that there was quite a high degree of similarity.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
I use json-ld in our site but schema not found in markup tool bar moz
i use json-ld in our site. structure data in test without any error but in test markup moz in our site schema is not found! Is it time for Google to identify the code to verify the schema?
Moz Bar | | shokoufezand0 -
Crawler triggering Spam Throttle and creating 4xx errors
Hey Folks, We have a client with an experience I want to ask about. The Moz crawler is showing 4xx errors. These are happening because the crawler is triggering my client's spam throttling. They could increase from 240 to 480 page loads per minute but this could open the door for spam as well. Any thoughts on how to proceed? Thanks! Kirk
Moz Bar | | kbates1 -
Duplicate content found in scan
On June 8th we ran a Moz Crawl on our site. We found 144 pages that were flagged with duplicate content.
Moz Bar | | StickyLife
Again on June 13th we ran another moz crawl on our site and found 137 pages that were flagged with duplicate content. Then one final scan on June 22nd with 161 pages of duplicate content. After comparing the 3 different scans I see that, without making any changes, pages that were not flagged as duplicate content are now being flagged as duplicate content. While at the same time, pages that were originally flagged as duplicate content are now no longer showing up with duplicate content. I could understand if we made some changes to these pages but no changes were made. For example: On the 8th this page was flagged as duplicate content - https://www.stickylife.com/star-magnet
On the 13th and 22nd it was not flagged as duplicate content but no changes were made to that page. For reference it was flagged as duplicate content with the following page: https://www.stickylife.com/baseball-glove-magnet This page was also Not changed or altered between between these dates. In addition, when Moz scans our site through our campaign every Friday the results do not match what we see when we do a manual scan. Moz's weekly scan only reveals 14 pages with duplicate content as opposed to the numbers you see above. Why such inconsistencies in the Moz Scans?0 -
Moz Forum Responses Crashing Outlook 365
Don't know if anyone else has this issue every time I get a forum response email and I try to open it it seizes up Outlook 365. It totally crashes, all I can do is to kill the process and restart Outlook!!
Moz Bar | | seoman100 -
Moz Crawl Test says pages have no internal links
Greetings, I am working on a website, https://www.nasscoinc.com, and ran a Moz Crawl Test on it. According to the crawl test, only 2 of the website's hundreds of pages are receiving internal links. When I run a similar test on the site using Screaming Frog, I see that most of the pages have at least one internal link. I'm wondering if anyone has seen this before with the crawl test; and there is a way to get the crawl test to see the internal links? Thanks!
Moz Bar | | TopFloor0 -
Moz Conversion Tracking
Is there a place here on Moz where goals and conversions can be set up or viewed? If the latter, does Moz just take the data from Analytics? Any information on this would be appreciated. Thanks,
Moz Bar | | xvpn9020 -
I requested a new crawl, this was done but my dashboard only shows the crawl done last week?
We recently moved our old website to a new CMS and structure. there have been some configuration errors and I needed to make some changes with things like canonical url's etc. However I need to check if these changes have made a difference and requested a new crawl through the crawl test page. I was emailed each time that a new crawl had been done but my reporting and dashboards still only show data from the last scheduled crawl. Regards Chris
Moz Bar | | LRQA-Marketing0 -
408 errors in crawl diagnostics
Best community, The Crawl Diagnostics Report of Moz gave our website a lot of 408 errors like below: <dl> <dt>Title</dt> <dd>408 : Error</dd> <dt>Meta Description</dt> <dd>408 Request Time-out</dd> <dt>Meta Robots</dt> <dd>Not present/empty</dd> <dt>Meta Refresh</dt> <dd>Not present/empty</dd> <dd>-----------------------------------------------------------------------</dd> <dd>The report has diagnosed a lot of these (around 320), even though we cannot reproduce the error (we cannot seem to find it ourself). </dd> <dd>2 questions relating to this: </dd> <dd>* Can you (the people of Moz) reproduce the errors manually? </dd> <dd>* Is it possible that it is a bug in the spider of Moz itself (too many spiders crawling at the same time)?</dd> </dl>
Moz Bar | | arjen.koedam0