Why is Google Webmaster Tools reporting a massive increase in 404s?
-
Several weeks back, we launched a new website, replacing a legacy system moving it to a new server. With the site transition, webroke some of the old URLs, but it didn't seem to be too much concern. We blocked ones I knew should be blocked in robots.txt, 301 redirected as much duplicate data and used canonical tags as far as I could (which is still an ongoing process), and simply returned 404 for any others that should have never really been there.
For the last months, I've been monitoring the 404s Google reports in Web Master Tootls (WMT) and while we had a few hundred due to the gradual removal duplicate data, I wasn't too concerned. I've been generating updated sitemaps for Google multiple times a week with any updated URLs. Then WMT started to report a massive increase in 404s, somewhere around 25,000 404s per day (making it impossible for me to keep up). The sitemap.xml has new URL only but it seems that Google still uses the old sitemap from before the launch. The reported sources of 404s (in WMT) don't exist anylonger. They all are coming from the old site.
I attached a screenshot showing the drastic increase in 404s. What could possibly cause this problem?
-
Thank you for both responses...
Nakul--
I have been following everything exactly as you have described. In general the goal during the development was to keep changes to an absolute minimum. This has not always been possible.
The majority of external links have been 301 redirected or in cases where the new server responds to two differnet URLs for the same content a canonical tag has been added.
I have noticed that 99% of the reported URLs are former internal links. The reported 404s are completely out of proportion (194k vs less than 5k pages in the new xml sitemap).
I am really worried. Is there anything else I can do beside monitoring and hopping?
How long does it typically take to for "Things have to work their way out of its system."?
Is it possible that Google is somehow accessing the old IP address (although the DNS records for the domain have changed)? We left the old server alive and planning to shut it down after the second site has been moved away from it.
Thanks,
Adam
-
Agreed; it could an after effect and stems from inbound URLs to your site from other sites. That's what the majority of the 404s I see in GWT come from (vs being bad pages within my site).
Google probably isn't using the old sitemap if you gave them a new one. What could be happening is that it still needs to "reorganize" and reconcile your old URLs and new URLs. The indexed pages don't just disappear overnight or get replaced immediately because of a site map change. Things have to work their way out of its system.
If there's specific URLs you want to try to remedy immediately, look into the GWT Remove URL option under the optimization section.
-
What I'd suggest doing is randomly revising some of those 404's that appear and check whether they should indeed be 404s. Are there any bulk rules / wildcard 301s you can implement to redirect the traffic for 3-6 months ?
These URLs are usually found from external links to your website. When you click on a detail of any of the reported 404s, it tells you what the error details are, whether this link is in the sitemap or where it is linked from. You'd realize in most cases it's linked from somewhere. If it's an internal link, correct it. If it's external, do you think the webmaster might update it if you contact them or is it easier to just set a 301, retaining the SEO value ?
I hope this helps.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to increase DA of website?
My website Domain Authority is not improving what should I do to improve DA of website
Intermediate & Advanced SEO | | JOkoiue0 -
Recovery after recent Google update
Hi guys. This is somewhat a continuation for this topic: https://moz.com/community/q/january-2016-massive-rankings-fluctuations After that update, several of our clients and our website as well have experienced high fluctuation rankings period, which ended up in huge drops - 10-20 spots. Which, obviously, made everyone unhappy. Anybody knows what exactly the change was about? What should we fix/take a look at, analyze again? We aren't using any shady techniques or black hat. Everything is honest. All metrics, number of backlinks etc are going up, no major changes have been recently made. Please, help!
Intermediate & Advanced SEO | | DmitriiK0 -
Google places Ad
Like adwords is their any good way of advertising a business place at Google map? If the answer is yes.Can you please take me through the process and give me rough idea about cost?
Intermediate & Advanced SEO | | csfarnsworth0 -
301 Redirect and Webmaster Central
I've been working on removing canonical issues. My host is Apache. Is this the correct code for my htaccess? RewriteEngine On
Intermediate & Advanced SEO | | spkcp111
RewriteCond %{HTTP_HOST} ^luckygemstones.com$ [NC]
RewriteRule ^(.*)$ http://www.luckygemstones.com/$1 [R=301,L] SECOND!!! I have two websites under Google's Webmaster Central; http://luckygemstones.com which gets NO 404 soft errors... AND http://www.luckygemstones.com which has 247 soft 404 errors... I think I should DELETE the http://luckygemstones.com site from Webmaster Central--the 301 redirect handles the"www" thing. Is this correct? I hate to hose things (even worse?) Help! Kathleen0 -
Google Preview not showing images
No matter which our pages i find in the SERP the Google Preview does not show images for www.spies.dk Try searching for "rejser til malaysia" on google.dk. The result www.spies.dk/malaysia is not showing images. Why is that? Using the "google preview tool" under labs in Google Webmaster Tools sys it found 13 errors on that page: | Ressource: | Information: http://images2.spies.dk/images/SiteID11/SiteLayout/logo-spies.png?v=4 | Indekseret http://images1.spies.dk/images/StaticLayout/Spies_hori_sRGB.png?v=1 | Indekseret http://images1.spies.dk/images/SiteID11/Button/search-button-text3.png?v=2 | Indekseret http://images2.spies.dk/images/Country/my1001_10_48.jpg?v=1 | Indekseret http://images2.spies.dk/images/SiteID11/Button/Opdater_bt.png?v=2 | Indekseret http://images1.spies.dk/images/SiteID11/Button/Sog_bt.png?v=3 | Indekseret http://images2.spies.dk/images/Country/my1002_2_22.gif?v=1 | Indekseret http://images2.spies.dk/images/Resort/bkikki1001_4_11.jpg?v=1 | Indekseret http://images2.spies.dk/images/SiteID11/Button/Vaelg_bt.png?v=7 | Indekseret http://images1.spies.dk/images/Resort/bkiskn1001_4_11.jpg?v=1 | Indekseret http://images1.spies.dk/images/Resort/kulkum1001_4_11.jpg?v=1 | Indekseret http://images1.spies.dk/images/Resort/lgklak1001_4_11.jpg?v=1 | Indekseret https://track.adform.net/serving/scripts/trackpoint/async/ | Indekseret |
Intermediate & Advanced SEO | | alsvik0 -
How does Google know if a backlink is good or not?
Hi, What does Google look at when assessing a backlink? How important is it to get a backlink from a website with relevant content? Ex: 1. Domain/Page Auth 80, website is not relevant. Does not use any of the words in your target term in any area of the website. 2. Domain/Page Auth 40, website is relevant. Uses the words in your target term multiple times across website. Which website example would benefit your SERP's more if you gained a backlink? (and if you can say, how much more would it benefit - low, medium, high).
Intermediate & Advanced SEO | | activitysuper0 -
How to increase the traffic ?
Hi Everyone, I am a bit a newbie in SEO and I read different articles and comments regarding the SEO but I am a bit stuck to get traffic through www.organicur.com. It's a really new website build through Prestashop (1-2 month). I used the tool keyword analysis to look after keywords not to competitive. I used the on-page optimization of Seomoz until to have A for every pages and I have started to build backlinks. But the traffic doesn't improve at all. Does that mean my keywords are not relevant enough ? Do I need to wait and carry on the links building. Do I need to go through PPC ? Thanks a lot for your reply, K
Intermediate & Advanced SEO | | NeSEO0 -
Any quality Local SEO tools with good reports?
We have used BrightLocal for some time now. With Bright Local we do not like the reports from a qualitative point of view in that the information is often wrong.
Intermediate & Advanced SEO | | RobertFisher
Does anyone recommend something other than Bright Local? Thanks,0