How should I go about repairing 400,000 404 error pages?
-
My thinking is to make a list of most linked to and most trafficked error pages, and just redirect those, but I don't know how to get all that data because i can't even download all the error pages from Webmaster Tools, and even then, how would i get backlink data except by checking each link manually?
Are there any detailed step-by-step instructions on this that I missed in my Googling?
Thanks for reading!!
-
Try using ScreamingFrog (app for mac and pc). It's free and will crawl your site and list all the errors an dresponse codes 9inc 404s) you can export these as a CSV file.
HTH
Steve
-
Thanks for responding!
The site wasn't deleted. I will have to investigate what caused all these 404s. I suspect a redesign.
Is there a way get a full list of 404 pages with backlink information if possible for each URL?
Thanks!
-
I'm guessing that you deleted a whole site without 301'ing.
What is the base URL? (myDomain.com)?
There is a chance that you could use an .HTACCESS file with some redirect rules to send whole batches to a new domain but these are all guesses without further detail from you.
Good luck...
Steve
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why Is this page de-indexed?
I have dropped out for all my first page KWDs for this page https://www.key.co.uk/en/key/dollies-load-movers-door-skates Can anyone see an issue? I am trying to find one.... We did just migrate to HTTPS but other areas have no problem
Intermediate & Advanced SEO | | BeckyKey0 -
Splitting down pages
Hello everyone, I have a page on my directory for example:
Intermediate & Advanced SEO | | SamBayPublishing
https://ose.directory/topics/breathing-apparatus The title on this page is small yet a bit unspecific:
Breathing Apparatus Companies, Suppliers and Manufacturers On webmaster tools these terms hold different values for each category so "topic name companies" sometimes has a lot more searches than "topic name suppliers". I was thinking if I could split the page into the following into three separate pages would that be better: https://ose.directory/topics/breathing-apparatus (main - Title: Breathing Apparatus)
https://ose.directory/topics/breathing-apparatus/companies (Title: Breathing Apparatus Companies)
https://ose.directory/topics/breathing-apparatus/manufacturers (Title: Breathing Apparatus Manufacturers)
https://ose.directory/topics/breathing-apparatus/suppliers (Title: Breathing Apparatus Suppliers) Two Questions: Would this be more beneficial from an SEO perspective? Would google penalise me for doing this, if so is there a way to do it properly. PS. The list of companies may be the same but the page content ever so slightly different. I know this would not effect my users much because the terms I am using all mean pretty much the same thing. The companies do all three.0 -
One page ranking for all key words, when other targeted pages not ranking
Hi everyone I am fairly new to SEO but have a basic understanding. I have a page that has a lot of content on it (including brand names and product types and relevant info) ranking for a quite a few key words. This is cool, except that I have pages dedicated to each specific key word that are not ranking. The more specific page still has a lot of relevant text on it too. eg. TYRES page - Ranks first for "Tyres". Ranks okay for many tyre key words, including "truck tyres"
Intermediate & Advanced SEO | | JDadd
TRUCK TYRES page - not ranking for "truck tyres" Further on, I then have pages not ranking all that well for more specific key words when they should. eg HONDA TRUCK TYRES - Then has a page full of product listings - no actual text. Not Ranking for "honda truck tyres". ABC HONDA TRUCK TYRE - not ranking for "abc honda truck tyre" key word
These pages don't have a lot of content on them, as essentially every single tyre is the same except for the name. But they do have text. So sometimes, these terms don't rank at all. And sometimes, the first TYRES page ranks for it. I have done the basic on page seo for all these pages (hopefully properly) including meta desc, meta titles, H1, H2, using key words in text, alt texting images where possible etc. According to MOZ they are optimised in the 90%. Link building is difficult as they are product listings, so other sites don't really link to these pages. Has anyone got ideas on why the top TYRES page might be so successful and out ranking more specific pages? Any ideas on how I can get the other pages ranking higher as they are more relevant to the search term? We are looking in to a website redesign/overhaul so any advice on how I can prevent this from happening on essentially a new site would be great too. Thanks!0 -
Links on page
Hi I have a web page which lists about 50-60 products which links out to either a pdf on the product or the main manufacturers website page containing product detail. The site in non e-commerce is this the site/page likely to get hit by Penguin? Would it be best to create a separate page for the product/manufacturer group i.e 5 or 6 pages but linking out to the PDFs etc...?
Intermediate & Advanced SEO | | Cocoonfxmedia0 -
Do search engines crawl links on 404 pages?
I'm currently in the process of redesigning my site's 404 page. I know there's all sorts of best practices from UX standpoint but what about search engines? Since these pages are roadblocks in the crawl process, I was wondering if there's a way to help the search engine continue its crawl. Does putting links to "recent posts" or something along those lines allow the bot to continue on its way or does the crawl stop at that point because the 404 HTTP status code is thrown in the header response?
Intermediate & Advanced SEO | | brad-causes0 -
Webmaster or analytics can we find pages that are 404
Hi, Webmaster or analytics can we find pages that are 404 404 landing pages? From which source to what page and from page i get the 404 error when someone accesses my webpage? Need to know which pages are live on sites that are broken in my site Thanks
Intermediate & Advanced SEO | | mtthompsons0 -
HTTP Status Bad Request - 404, but also, add a 400 HTTP Status in certain circumstances?
We currently have a custom 404 page set up for our clients, but the developer has it returning a HTTP 200 for the status code. Big no, no. I'm having that fixed right now. My question is, currently, the custom 404 page is only returned for urls with the extension .aspx: For example : ilovepizza.com/pepperni.aspx would return a 404 page because the correct page is ilovepizza.com/pepperoni.aspx Any other format of URL without the extension (example ilovepizza.com/thumbtack) does not trigger the custom 404 page we've created, but it does trigger a server error with a 404 HTTP status page. I want to change this so this type of error also triggers the custom 404 page because it's more user-friendly and would return them to the website. My question: Is there any benefit to making the /thumbtack errors return the custom 404 page but with a 400 Bad Request HTTP Status? Kind of a novice here in those aspects, but does the 400 Bad Request status indicate that it was a user mistake and not a mistake created on the website? Other suggestions?
Intermediate & Advanced SEO | | EEE30 -
List of Off page techniques
Hello Everyone, Please share the list of off page techniques to improve ranking and which techniques completely avoided?
Intermediate & Advanced SEO | | Alick3000