Thousands of 503 Errors
-
I was just checking Google Webmaster Tools for one of the first times (I know this should have been a regular habit).
I noticed that on Feb 8th we had almost 80K errors of type 503. This is obviously very alarming because as far as I know our site was up and available that whole day. This makes me wonder if there is a firewall issue or something else that I'm not aware of.
Any ideas for the best way to determine what's causing this?
Thanks,
Chris
-
Cyrus,
Thanks for the props, but also cool on the crawl delay link. I wish I could say I knew about it before this answer, but I didn't; cool stuff for bigger high update sites.
Always appreciate what you have to say as I learn a lot from you.
Best
-
Hi Chris,
This is a really hard problem to diagnose from the outside like, so I'll just give you my thoughts.
1. Are the URLs throwing the 503 errors real pages? Can they be accessed normally by human visitors through the site? I only mention this because sometimes you get software generating a bunch of random links that go nowhere, and weird stuff starts to happen when Google crawls those URLs. Normally you'd see these result as 404s, however.
2. Is the date in Google Webmaster Tools for the 503 errors recent? Sometimes they log those for a long time after the problem is actually solved, especially for URLs they don't visit much.
3. How often does your site go down?
4. Try performing a "Fetch as Googlebot" test on some of the effected URLs
5. I doubt googlebot is crashing your site, but you could always try a crawl delay
6. If nothing else, you'll find the problem at the serving/hosting level. Can't be much help there, unfortunately.
-
It turns out that the Magento patch did NOT fix the problem. We are still receiving tens of thousands of 503 errors when Googlebot requests a page. The site is not down. I can look in the access_log and see that the request was responded to with a 503 error.
Any ideas? This has to be killing our chances for organic traffic until this gets resolve.
-
Hi Robert,
Thanks for the response. It turns out that this is due to a bug in our hosting software, Magento, that results in googlebot not being handled correctly. Apparently there's a patch that's being tested now.
Thanks,
Chris
-
Do you know where you are hosted? Have you called them to see if the server is down or intermittently down?
Here is a how to resolve link.
Look at bottom and follow the directions regarding using the wayback machine to see if it is temporary or the server is down for maintenance.
That given, if you give us a url, it is easier to assist you.
Best, let us know.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
404 Errors for Form Generated Pages - No index, no follow or 301 redirect
Hi there I wonder if someone can help me out and provide the best solution for a problem with form generated pages. I have blocked the search results pages from being indexed by using the 'no index' tag, and I wondered if I should take this approach for the following pages. I have seen a huge increase in 404 errors since the new site structure and forms being filled in. This is because every time a form is filled in, this generates a new page, which only Google Search Console is reporting as a 404. Whilst some 404's can be explained and resolved, I wondered what is best to prevent Google from crawling these pages, like this: mydomain.com/webapp/wcs/stores/servlet/TopCategoriesDisplay?langId=-1&storeId=90&catalogId=1008&homePage=Y Implement 301 redirect using rules, which will mean that all these pages will redirect to the homepage. Whilst in theory this will protect any linked to pages, it does not resolve this issue of why GSC is recording as 404's in the first place. Also could come across to Google as 100,000+ redirected links, which might look spammy. Place No index tag on these pages too, so they will not get picked up, in the same way the search result pages are not being indexed. Block in robots - this will prevent any 'result' pages being crawled, which will improve the crawl time currently being taken up. However, I'm not entirely sure if the block will be possible? I would need to block anything after the domain/webapp/wcs/stores/servlet/TopCategoriesDisplay?. Hopefully this is possible? The no index tag will take time to set up, as needs to be scheduled in with development team, but the robots.txt will be an quicker fix as this can be done in GSC. I really appreciate any feedback on this one. Many thanks
Technical SEO | | Ric_McHale0 -
60,000 404 errors
Do 404 errors on a large scale really matter? I'm just aware that I now have over 60,000 and was wondering if the community think that I should address them by putting 301 redirects in place. Thanks
Technical SEO | | the-gate-films0 -
Wordpress 404 Errors
Hi Guys, One of my clients is scratching his head after a site migration. He has moved to wordpress and now GWT is creating weird and wonderful strange 404 errors. For example http://www.allsee-tech.com/digital-signage-blog/category/clients.html There are loads like the above which seem to be made up out of his blog and navigation http://www.allsee-tech.com/clients.html works! Any ideas? Is it a rogue plugin? How do we fix? Kind Regards Neil
Technical SEO | | nezona0 -
Bogus Crawl Errors in Webmaster Tools?
I am suddenly seeing a ton of crawl errors in webmaster tools. Almost all of them are URL links coming from scraper sites.that I do not own. Do you see these in your Webmaster Tools account? Do you mark them as "fixed" if they are on a scraper site? There are waaaay too many of these to make redirects. Thanks!
Technical SEO | | EGOL0 -
404 errors and what to do
Hi, I am fairly new to the whole seo thing and am still getting confused a bit as to what to do to sort things out. I've checked the help pages but I cannot seem to find the issue. I've just signed up so my site is crawled for the first time and coming up with more then a 1000 404 errors. I checked a couple of the links via the report I downloaded and it does indeed show a 404 error but when I check the pages all seems to work fine. I did find one issue where an image if clicked on twice was pointing to an url with 'title= at the end. Now I have tried to get of that but couldn't find anything wrong. I'm a bit lost as to where to start!
Technical SEO | | junglefrog0 -
Robots.txt file getting a 500 error - is this a problem?
Hello all! While doing some routine health checks on a few of our client sites, I spotted that a new client of ours - who's website was not designed built by us - is returning a 500 internal server error when I try to look at the robots.txt file. As we don't host / maintain their site, I would have to go through their head office to get this changed, which isn't a problem but I just wanted to check whether this error will actually be having a negative effect on their site / whether there's a benefit to getting this changed? Thanks in advance!
Technical SEO | | themegroup0 -
404 Error on Spider Emulators
I recently began working at a company called Uncommon Goods. I ran a few different spider emulators on our homepage (uncommongoods.com) and I saw a 404 Error on SEO-browser.com as well as URL errors on Summit Media's emulator and SEOMoz's crawler. It seems there is a serious problem here. How is this affecting our site from an SEO standpoint? What are the repercussions? Also, I know we have a lot of javascript on our homepage..is this causing the 404? Any advice would be much appreciated. Thanks! -Zack
Technical SEO | | znotes0 -
Google webmasters shows 37K not found errors
Hello we are using Joomla as our cms, months ago we used a component to create friendly urls, lots of them got indexed by google, testing the component we created three different types of URL, the problem now is that all of this tests are showing in google webmasters as 404 errors, 37,309 not found pages and this number is increasing everyday. What do you suggest to fix this?? Regards.
Technical SEO | | Zertuxte0