Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
Why Do Different Tools Report 404s Differently?
-
Hi Mozers,
How come Moz reports just six 404 errors, whereas Google Search Console reports 250 and Screaming Frog only reports a dozen? It seems to me that these results are all over the place. Shouldn't these reports be more consistent?
I do understand that Search Console includes historical data and that URLs or issues need to be "marked as fixed" in order for them to go away, however, even if I do this, Google ends up reporting far more errors than anything else.
Do 404s reported by Moz and Screaming Frog NOT include external links? It seems to me that this could be partially responsible for the issue.
Also, is there a way to efficiently track the source of the 404s besides clicking on "Linked From" within Search Console 250 times? I was looking for something like this is Moz or SF but no luck.
Any help is appreciated.
Thanksabunch!
-
One more sidenote - you can also use inbound link auditing tools like Moz Open Site Explorer, AHREFS, or especially Cognitive SEO to collect as many of your incoming links as possible, then filter them for which ones 404 and then you'll know which external pages contain the broken incoming links. CognitiveSEO is especially well set up to do this, but it's not a cheap tool (Does have a free trial though).
Hope that helps?
Paul
-
While essential to maintaining the quality of the website, remember this process will only show missing URLs as detected from within the website. It can't show any info about all the broken links that may exist externally on other websites.
P.
-
These tools are designed to report very different things, Eric. SF and Moz are only able to tell you what they can discover about what's happening within your site, since that's all they have access to. (Moz's link index isn't tied into their site crawl reports). So your own pages that are linking to other pages which no longer exist.
Search Console is also showing what's happening on the rest of the web outside your website. So it's picking up and reporting on links on other websites that it's trying to follow but finding 404 at the end on your site.
So, for example, Search Console is the only one that's going to be able to find and report on URLs that you used to have, but that haven't been correctly redirected after the URLs changed, like in a redesign. Assuming you cleaned up your internal linking after the redesign, the only way to know that these other URLs still haven't been properly redirected is via GSC's external visibility.
If GSC is finding a lot of these, and you want to proactively find as many more as possible without waiting for GSC to find them. there are a couple of methods available to try to address in bulk. One way is to run a Screaming Frog crawl of the previous version(s) of the site in the Wayback Machine, then rerun those as a list to discover which ones now 404. Or if you have an old sitemap, you can upload that into SF and run that and check for 404s.
A partial alternative is to run a custom report in GA that shows the referrer to the 404 page. But it will only track actual clickthroughs, whereas GSC actually reports on external broken links, whether they've been clicked on recently or not. So much broader and a bit more proactive. Eoghan at Rebelytics has a great post on assembling these useful reports, including links to directly import from the Solutions Gallery. Do note though, this data can often be pretty "dirty" with direct/(not set) referrers, or very vague ones like "Facebook"
But bottom line - use your Screaming Frog crawl to find and fix the broken links within your own site. USe the GSC list (downloaded) to write appropriate redirects for the URLs it finds that haven't' been covered by your internal fixes. Since you can't do anything about the broken links on third-party sites, the only control you have of them is writing redirects inside your own system. Put another way - it's not worth the effort to find where on the external webs the broken inbound links are from, since you can't do anything about changing them anyway.
Hope that all makes sense?
Paul
P.S. Marking the 404s as fixed in GSC will suppress them for about 90 days, but if the broken links still exist out on the web or in the index and are re-crawled, they will again start to show up in your GSC list until they've been crawled often enough that G decides to trust the 404 and may stop crawling them. Once you apply redirects, G will stop reporting them as 404s. Do note though that if the equivalent content really is gone, there's nothing wrong with letting the broken links 404. Trying to artificially redirect them to unrelated pages (like the home page) will just result in G treating them as (soft)404s anyway.
-
The answer to this is actually quite simple, Moz and Google have data that's not as up to date as Screaming Frog, so Moz and Google can be reporting historical issues that you've resolved allowing them to have larger numbers reported. The other issue is sample size, so Moz for example may not have your complete site in their index, yet Google does.
"Also, is there a way to efficiently track the source of the 404s besides clicking on "Linked From" within Search Console 250 times? I was looking for something like this is Moz or SF but no luck."
As for that question, easiest way is through Screaming Frog as you're able to get "live" checks on your site (every time you crawl your site again). If you look at the attached image you'll see that I'm on the 404 response code, I've clicked on the URL I want to know where it's being linked from (Inlinks) and I've got a list of pages and then other information such as anchor/alt text.
Hopefully this helps you out.
Tom
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Difference between urls and referring urls?
Sorry, nit new to this side of SEO We recently discovered we have over 200 critical crawler issues on our site (mainly 4xx) We exported the CSV and it shows both a URL link and a referring URL. Both lead to a 'page not found' so I have two questions? What is the difference between a URL and a referring URL? What is the best practice/how do we fix this issue? Is it one for our web developer? Appreciate the help.
Moz Pro | | ayrutd1 -
What impact has external links reported as http503
Hello I have 72 external broken links who are reported with a http 503 status. When I ask the owner of that site, he confirmed he has removed it from GA by using this status. My question is. Does this have an impact for the quality of my site? The site is available, there is no delay when you open the link. Thank you for your support. br
Moz Pro | | kjerstibakke
Kjersti Bakke0 -
Best tools for an initial website health check?
Hi,
Moz Pro | | CamperConnect14
I'd like to offer free website health checks (basic audits) and am wondering what tools other people use for this? It would be good to use something that presents the data well. Moz is great but it gets expensive if I want to offer these to many businesses in the hope of taking on just a few as clients and doing a full manual audit for them. So far I've tried seositecheckup.com (just checks a single page though), metaforensics.io and mysiteauditor. Thanks!0 -
Why is MOZ and Google search Volume so different?
A search term in MOZ shows the monthly search volume to be 49K. In Google, the same term shows the search volume at only 1300 monthly searches. Which do I trust? Thanks, Don
Moz Pro | | rcman0 -
Redirect analysis tool
I'm looking for a tool like this: http://www.internetofficer.com/seo-tool/redirect-check/ that can check hundreds/thousands of URLs and give me a report as to which ones have been redirected. Does anyone know of something that can do this?
Moz Pro | | glass010 -
Colors in keyword Difficulty Report
Hi Everyone Two quick questions today 1. How can I find out hat the different colors within the keyword Difficulty Report represent and how can I see examples of how this information can help us with our data analysis? 2. The second question I have is regarding the Term Extractor. Seems when I ran a domain it provided the wrong data. For example, it stated that a certain keyword exists certain number of times within the description and title of the page but when I looked at the source this was not the case so it made studying the competition harder.
Moz Pro | | DRTBA
Any suggestions or has anyone else noticed this? Thanks in advance for all your help.0 -
Discrepancies in PA and LRDs reported in different SEOmoz tools
I've noticed a difference in the reported PA and LRD numbers for URLs depending on whether you use Open Site Explorer, or look at the same metrics from within the rankings history (in your campaign set up). I've checked this for a few URLs and what I'm seeing is the reported scores for PA and LRDs is different 9 times out of ten. The PA is sometiomes higher on one report, lower on another, or vice versa. Same for LRDs. I thought it might be because one report was lagging behind and using old data, but that would only make sense if I was seeing an increase in reported LRDs, but it just as often shows a decrease ! Is this just a bug in the campaign>rankings history report or is there a reason for the discrepancies?
Moz Pro | | Websensejim0 -
A tool to submit websites in directories
Hello I am looking for a tool to help me to submit websites in directories, something like the yooda tool. http://www.yooda.com/outils_referencement/submit_center_yooda/ This tool seems good no? do you offer something similar at seomoz? or where could I find some similar tools and in which languages is it available?
Moz Pro | | bigtimeseo2