Get rid of a large amount of 404 errors
-
Hi all,
The problem:Google pointed out to me that I have a large increase of 404 errors. In short I had software before that created pages (automated) for long tale search terms and feeded them to google. Recently I quit this service and all those pages (about 500000) were deleted. Now google GWM points out about 800000 404 errors.
What I noticed: I had a large amount of 404's before when I changed my website. I fixed it (proper 302) and as soon as all the 404's in GWM were gone I had around 200 visitors a day more.
It seems that a clean site is better positioned.
Anybody any suggestion on how to tell google that all urls starting with www.domain/webdir/ should be deleted from cache?
-
Hi,
You have several options:
1. Do nothing as with the next or the third google crawl those pages will be drooped from the index (google's index / cache)
2. Set those pages to 410 instead of 404 - that mean page no longer exists so google will clear it's cache when the 410 will be discovered (next google's crawl cycle)
3. Request the folder to be removed via your Web master tool account and then block it via robots.txt
http://googlewebmastercentral.blogspot.com/2010/03/url-removal-explained-part-i-urls.html
Hope it helps.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to get rid of bot verification errors
I have a client who sells highly technical products and has lots and lots (a couple of hundred) pdf datasheets that can be downloaded from their website. But in order to download a datasheet, a user has to register on the site. Once they are registered, they can download whatever they want (I know this isn't a good idea but this wasn't set up by us and is historical). On doing a Moz crawl of the site, it came up with a couple of hundred 401 errors. When I investigated, they are all pages where there is a button to click through to get one of these downloads. The Moz error report calls the error "Bot verification". My questions are:
Technical SEO | | mfrgolfgti
Are these really errors?
If so, what can I do to fix them?
If not, can I just tell Moz to ignore them or will this cause bigger problems?0 -
Big Increase in 404 Errors after Google Custom Search Engine Install on Website
My URL is: http://www.furniturefashion.comHi forum.I recently installed a Custom Google Search Engine (https://www.google.com/cse/) on my blog about ten days ago. Since then my 404 errors in Webmaster Tools has skyrocketed by several thousand. I had not had an issue before. Once it was installed the 404 errors started appearing. What's interesting is that all the errors have the URL then the word "undefined" at the end. I have attached a screen shot from my Webmaster Tools dashboard. Also, there are a few examples below of what the URLs are that have the 404 errors.wood_closet_organizer_to_improve_space_utilization/undefinedsmall-sweet-10-inspiring-small-kitchen-designs/undefined Has anyone had this issue? I very much want the search engine on my site, but not at the expense of several thousand 404 errors. My site queries has been going down since the installation of the custom search engine. Here is some of the code that I have below that I took off my site doing a "view source". Any help would be greatly appreciated.href='http://cdn.furniturefashion.com/wp-content/plugins/google-custom-search/css/smoothness/jquery-ui-1.7.3.custom.css?ver=3.9.2' type='text/css' media='all' />rel='stylesheet' id='gsc_style_search_bar-css' href='http://www.google.com/cse/style/look/minimalist.css?ver=3.9.2' type='text/css' media='all' />rel='stylesheet' id='gsc_style_search_bar_more-css' href='http://cdn.furniturefashion.com/wp-content/plugins/google-custom-search/css/gsc.css?ver=3.9.2' type='text/css' media='all' />< uXRSEkC
Technical SEO | | will21120 -
404 or 503 Malware Content ?
Hi Folks When it comes to malware , if I have a site that uses iframe to show content off 3rd party sites which at times gets infected. Would you recommend 404 or 503 ing those pages with the iframe till the issue is resolved ? ( I am inclined to use 503 .. ) Then take the 404/503 off and ask for a reindex ( from GWT malware section ) OR Ask for a reindex as soon as the 404/503 goes up. ( I do understand we are asking to index as non existing page , but the malware warning gets removed ) PS : it makes sense for this business to showcase content using iframe on these special pages . I do understand these are not the best way to go about SEO.
Technical SEO | | Saijo.George0 -
Getting listed in DMOZ after a red mark against me?
Hi everyone, Before I start, I want to point out that I realise DMOZ is not everything. If I can't get a link, then I'll accept it, but I'd like to try damn hard to get a link in there if possible as I honestly believe we are a reputable company that deserves to be in there with other companies in our region. So, lets go back to the start: A few years ago, when I was young and foolish, I posted a request on rentacoder (now vworker) asking for someone to submit us to DMOZ for $xxx. If I remember correctly, we accepted someone for the job and they submitted our site. Since then, we've never been accepted. I tried submitting the site a couple of years ago and realised it still hadn't been submitted around a year ago. I thought this was a bit strange, so I did a bit of digging. I ended up talking to someone who was a DMOZ editor. They looked at my listing on DMOZ and basically said that I'd be rejected for 'bribing an editor' - something that I still maintain I didn't do. Never at any point have I said to someone that I would pay them to guarantee us a listing. At the time, I thought there was a 'certain way' to write a submission, but now that I've grown up a bit, I've realised it's fairly easy to get a listing if you write a well written description and are honest - I've done this for many clients sites which are now accepted. Unfortunately, this does mean that It looks like I won't be able to get our agency on DMOZ, but I just wanted to give one last push and see if anyone on SEOMoz knows of any one I could speak to in order to lift this ban? I realise now that it was probably a stupid thing to do, and in hindsight, I should have just waited like everyone else. I hold my hands up to that and I've worked over the last few years to grow our agency into something special for our clients. As I say, I realise dmoz isn't everything, but it is a directory that still holds value and I would like to be linked off there if possible. Any tips on how I may be able to beg for my ban to be lifted would be much appreciated. Kind regards, Lewis PS: I applied to become an editor at one point, this was when I realised something wasn't right - http://www.resource-zone.com/forum/index.php?showtopic=52658
Technical SEO | | PinpointDesigns0 -
404 not working?
Apologies - not strictly a search question, although I am concerned that it may be negatively impacting my sites. If I enter an invalid URL - the "page not found 404 error" doesn't seem to be working. I can enter http://www.%sitename%/index.php/randomnamemadeup/ and it displays the homepage (albeit with a broken CSS). I can't fathom it out - any help appreciated. Thanks, David
Technical SEO | | newstd1000 -
Duplicate page error
SEO Moz gives me an duplicate page error as my homepage www.monteverdetours.com is the same as www.monteverdetours.com/index is this actually en error? And is google penalizing me for this?
Technical SEO | | Llanero0 -
Google's "cache:" operator is returning a 404 error.
I'm doing the "cache:" operator on one of my sites and Google is returning a 404 error. I've swapped out the domain with another and it works fine. Has anyone seen this before? I'm wondering if G is crawling the site now? Thx!
Technical SEO | | AZWebWorks0 -
Duplicate content error - same URL
Hi, One of my sites is reporting a duplicate content and page title error. But it is the same page? And the home page at that. The only difference in the error report is a trailing slash. www.{mysite}.co.uk www.{mysite}.co.uk/ Is this an easy htaccess fix? Many thanks TT
Technical SEO | | TheTub1