Crawl Diagnostics Report Lacks Information
-
When I look at the crawl diagnostics, SEOMoz tells me there are 404 errors.
This is understandable, because some pages were removed.
What this report doesn't tell me is how those pages were discovered.
This is a very important piece of information, because it would tell me there are links pointing to those pages, either internal or external. I believe the internal links have been removed.
If the report told me how if found the link, I would be able to take immediate action. Without that information, I have to go so a lot of investigation. And when you have a million pages, that isn't easy.
Some possibilities:
- The crawler remembered the page from the previous crawl.
- There was a link from an index page - i.e. it is in the database still
- There was an individual link from another story - so now there are broken links
- Ditto, but it in on a static index page
- The link was from an external source - I need to make a redirect
Am I missing something, or is this a feature the SEO Moz crawler doesn't have yet?
What can I do (other than check all my pages) to discover this?
-
OK thank you, Ralph
I can work on that.
-
I think it's the SEOMoz crawler, but what I have found is that the error reports are limited here whereas GWT is much bigger and shows the links leading to the error. My guess is that SEOMoz limit the number of crawl errors they show due to limitations set on their crawler i.e. while their crawl is comprehensive, it's not going to capture what Google does.
-
Thank you Ralph.
Yes, had it for years. So is this a GWT report? I thought it was SEOMoz !
No not IIS, Linux.
-
If you download the csv file for the crawl you can sort it by http status to get all of the 404 errors together. Then there is a specific column that contains the referrer that provides the information you are after.
-
This may be a silly question, but have you got Google Webmaster tools installed? That will show you the source of the errors.
If your site is on IIS then you should also use the awesome IIS SEO toolkit provided by Microsoft for free.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
I need an interlinking report for my site, is there a report in Moz or another application that tell me how all of my pages are linked to other pages on my site?
I am in the process of doing a redesign for one of my sites. I need an interlinking report for my site. Is there a report in Moz or another application that tell me how all of my pages are linked to other pages on my site?
Moz Pro | | seoflorida0 -
404 : Errors in crawl report - all pages are listed with index.html on a WordPress site
Hi Mozers, I have recently submitted a website using moz, which has pulled up a second version of every page on the WordPress site as a 404 error with index.html at the end of the URL. e.g Live page URL - http://www.autostemtechnology.com/applications/civil-blasting/ Report page URL - http://www.autostemtechnology.com/applications/civil-blasting/index.html The permalink structure is set as /%postname%/ For some reason the report has listed every page with index.html at the end of the page URL. I have tried a number of redirects in the .htaccess file but doesn't seem to work. Any suggestions will be strongly appreciated. Thanks
Moz Pro | | AmanziDigital0 -
SEO on-demand crawl
what happened to the on-demand crawl you could do in PRO when they switched to the new MOZ site?
Moz Pro | | Vertz-Marketing0 -
Ranking report orderng when exporting to pdf
When I am in my rankings report I like to view the results in order of ranking, so that the keywords with higher position in google for example appear at the top, which is great. However when I export it to PDF it seems to default back to appearing in alabetical order... is there a way around this? Thanks!
Moz Pro | | stardigital0 -
Incomplete historical ranking reports
Hei there, I tried to update my datas using your historical ranking reports, but last week the reports were incomplete not having crawled all keywords... Did I do somethign wrong or is that a mistake on oyur side? thanks for hleping me Sarah
Moz Pro | | SEOKrauteratwork0 -
Report on a whole domain
Hi, I am wondering if I can run a report which gives me the OSE results for every page on a domain, as opposed to just a single page? Can't find this anywhere! thanks, Brian
Moz Pro | | VISISEEKINC0 -
SEOMOZ reports the statistics.
SEOMOZ reports the Statistics, but where do i manage & improve??? Simply Statistics is all about SEOMOZ??
Moz Pro | | webicers0 -
On-Page Summary (Report Cards) automation?
Hi everyone, Under the "On-Page" tab which shows your report cards, is there a way of getting it to grade your entire site? One of my site's is only ~20 pages so it's no big deal to manually enter each URL and set each one to update weekly. But what if I have a site that has ~1,000 pages and I want to optimise each and every page for my main keyword using the report cards feature? Thanks in advance! 🙂 Ash
Moz Pro | | AshSEO20110