Crawl Diagnostics Report
-
I'm a bit concerned about the results I'm getting from the Crawl Diagnostics Report.
I've updated the site with canonical urls to remove duplicate content and when I check the site - it all displays the right values, but the report, which has just finished crawling is still showing a lot of pages as duplicate content.
Simple example:
Both of them are in the duplicate content section although both have canonical url set as:
Does each crawl check the entire site from the beginning or just the pages it didn't have a chance to crawl the last time?
This is just one of 333 duplicate content pages, which have canonical url pointing to the right page.
Can someone please explain?
-
Yep!
That's why I really like the csv files because you can sort stuff and filter things down to specifically what you want to see.
-
Hi Kenny,
Thanks for getting back to me.
So is it just the way it is reported on the page and it's not the actual problem with the duplicate content?
-
Hi Sebastian,
Sorry for the confusion. Our software currently reports those urls as having both duplicate content and canonical tags. I find that the best way to view this information is by exporting your crawl diagnostic's csv file. You can easily locate the export functionality in the upper right of the crawl diagnostic page.
Kenny
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
WEbsite cannot be crawled
I have received the following message from MOZ on a few of our websites now Our crawler was not able to access the robots.txt file on your site. This often occurs because of a server error from the robots.txt. Although this may have been caused by a temporary outage, we recommend making sure your robots.txt file is accessible and that your network and server are working correctly. Typically errors like this should be investigated and fixed by the site webmaster. I have spoken with our webmaster and they have advised the below: The Robots.txt file is definitely there on all pages and Google is able to crawl for these files. Moz however is having some difficulty with finding the files when there is a particular redirect in place. For example, the page currently redirects from threecounties.co.uk/ to https://www.threecounties.co.uk/ and when this happens, the Moz crawler cannot find the robots.txt on the first URL and this generates the reports you have been receiving. From what I understand, this is a flaw with the Moz software and not something that we could fix form our end. _Going forward, something we could do is remove these rewrite rules to www., but these are useful redirects and removing them would likely have SEO implications. _ Has anyone else had this issue and is there anything we can do to rectify, or should we leave as is?
Moz Pro | | threecounties0 -
In Crawl Diagnostics, length of title element is incorrect
Hey all, It appears the Moz crawler is misreading the number of characters in my website's page titles. It shows 72 characters for the following page's title element: http://giavan.com/products/orange-crystal-chain-necklace-with-drop The page title for this web page is: Orange Crystal Chain Necklace with Drop | Giavan which is 48 characters. As it stands, this page title is displayed at 48 characters in Google SERPs. I am getting "This Element is Too Long" issue on 925 pages, which is just about the entire site. These issues appeared after I added additional Shopify (Liquid) code to the page title. If you inspect the code, you will see title element looks a bit odd with extra spacing and line breaks. What I'd like to know is whether or not it's necessary to rewrite the Shopify code, for SEM purposes. My feeling is that it's okay because the page titles look fine in SERPs but those 925 Moz crawl errors are kind of scary. Thanks for your help!
Moz Pro | | RichAlbanese0 -
Unable to view crawl test
After doing a crawl test i get a download report. It then downloads in csv form and when I go to view it there is a curruption error or just a load of gibberish signs Can I not see the report onsite?
Moz Pro | | hantaah0 -
Rankings Report - figures clarifiction
I've noticed a possible discrepancy with my data and the figures in the Ranking Report >> Rankings which leads to this question: In top 3 : 10 On first page: 19 Is this meant to mean 10 of the 19 are in the top 3, or after the 10 top 3 there are 19 more keywords on first page?
Moz Pro | | MickEdwards0 -
Is SeoMOZ Crawl Diagnostics wrong here?
We've been getting a ton of critical errors (about 80,000) in SeoMoz' Crawl Diagnostics saying we have duplicate content in our client's E-commerce site. Some of the errors are correct, but a lot of the pages are variations like: www.example.com/productlist?page=1 www.example.com/productlist?page=2 However, in our source code we have used rel="prev" and rel="next" so in my opinion we should be alright. Would love to hear from you if we have made a mistake or if it is an error in SeoMoz. Here's a full paste of the script:
Moz Pro | | Webdannmark0 -
Is there a report that shows how many times your keywords are searched?
I see the keyword ranking report but I need a report that shows the number of searches on those keywords, Thanks!
Moz Pro | | AFSB0 -
Crawled pages are missing and showing just 1 page crawled
One of my campaign has got around 8500 pages crawled(seomoz) and reports are shown, but suddenly it is showing 1 page crawled. Why it is happened like this? How can i get back the previous reports?
Moz Pro | | Sulekha0 -
My Campaign has been crawling for about a week now
Can anyone tell me why one of my campaigns has been stuck in crawl mode for about a full week and it is still not done?!?!
Moz Pro | | nazmiyal0