The crawl report shows a lot of 404 errors
-
They are inactive products, and I can't find any active links to these product pages. How can I tell where the crawler found the links?
-
That's too easy Keri.
-
If you download the CSV of the report, there is a column that will list the referring URL for the 404.
-
Mike's suggestion is a great one. SF gives some great data that's easy to play with in Excel. The free version only crawls 500 pages, so if you have a small site, you'll probably get what you need. (I got 30k+ pages, so I use the paid version that's only about $160/yr.)
-
There is nothing wrong with having a "404 error" if the page is an expired product. Obviously, you don't wont to be linking to a 404 page on your site, so I'd suggest using a tool like Screaming Frog, potentially even OSE, and monitoring your 404 pages in Google Webmaster Tools to see if it is still currently being linked to.
If the page has external links pointing to it, I'd recommend 301 redirecting it to whatever category/subcategory the product belongs to.
-
Do a scan with Screaming Frog. It can point out exact pages where you are having broken links.
They have a free trial and a paid version.
This program is great for diagnosing many different website related issues.
Hope this helps.
Mike
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why my site not crawl?
hi all me allow rogerbot in robots.txt but rogerbot can't crawl my site my site: toska-co.ir
Moz Pro | | jahanidawodi0 -
WEbsite cannot be crawled
I have received the following message from MOZ on a few of our websites now Our crawler was not able to access the robots.txt file on your site. This often occurs because of a server error from the robots.txt. Although this may have been caused by a temporary outage, we recommend making sure your robots.txt file is accessible and that your network and server are working correctly. Typically errors like this should be investigated and fixed by the site webmaster. I have spoken with our webmaster and they have advised the below: The Robots.txt file is definitely there on all pages and Google is able to crawl for these files. Moz however is having some difficulty with finding the files when there is a particular redirect in place. For example, the page currently redirects from threecounties.co.uk/ to https://www.threecounties.co.uk/ and when this happens, the Moz crawler cannot find the robots.txt on the first URL and this generates the reports you have been receiving. From what I understand, this is a flaw with the Moz software and not something that we could fix form our end. _Going forward, something we could do is remove these rewrite rules to www., but these are useful redirects and removing them would likely have SEO implications. _ Has anyone else had this issue and is there anything we can do to rectify, or should we leave as is?
Moz Pro | | threecounties0 -
How to remove 404 pages wordpress
I used the crawl tool and it return a 404 error for several pages that I no longer have published in Wordpress. They must still be on the server somewhere? Do you know how to remove them? I think they are not a file on the server like an html file since Wordpress uses databases? I figure that getting rid of the 404 errors will improve SEO is this correct? Thanks, David
Moz Pro | | DJDavid0 -
Keyword Rankings Report Accuracy
How many of you routinely have inaccurate data in your Moz Pro keyword rankings reports? I just checked 5 of our terms that came in this morning - yes, it's a not logged in, non-personalized, incognito, cleared cache search - and none of them actually ranked where Moz said they ranked. One was listed in the top 5 and wasn't even on the first page. One was listed at position 3 but was actually at position 8, a big difference when it comes to CTR. And the report will have stuff like our brand name not ranked at all one week, then jumping by 45+ positions the next week, then gone the next week. And it doesn't fluctuate like that. I get that the reports are general to what most people see, but should such big disparities be expected?
Moz Pro | | Kingof50 -
Duplicate titles reported with canonical
Hi Mozzers, In the reports it is saying that I have some duplicate content and titles even though there is a canonical tag on them, is anyone else getting this?
Moz Pro | | KarlBantleman0 -
How often does seomoz crawl the site? Can you force a crawl at a specific time ?
How often does seomoz crawl the site? Can you force a crawl at a specific time ?
Moz Pro | | stewbuch18720 -
Duplicate Content and Titles in SEOMoz reports
I've had to rename some of the pages on my site and also move them to different locations. I placed a rel="canonical" on the old page pointing to the new one. The reports on my PRO Dashboard are telling me that I have Duplicate Content and Page Title errors. Do the SEOMoz automated reports take the rel="canonical" link into consideration or do I need to remove these pages and do a 301 redirect from the old to the new page?
Moz Pro | | TRICORSystems0 -
Personalised PRO Dashboard reports?
Hi guys, I'm loving the SEOMoz PRO dashboard and have learnt so much from the campaigns I've been running. I was wondering if I could personalise the reports to have my logo on them instead of SEO MOz's? At the moment, I export them and try recreate them offline, but it wouold be way easier if I could just send them the reports PDF'ed up straight from the dashboard.
Moz Pro | | Lindsay_D0