5XX (Server Error) on all urls
-
Hi
I created a couple of new campaigns a few days back and waited for the initial crawl to be completed. I have just checked and both are reporting 5XX (Server Error) on all the pages it tried to look at (one site I have 110 of these and the other it only crawled the homepage). This is very odd, I have checked both sites on my local pc, alternative pc and via my windows vps browser which is located in the US (I am in UK) and it all works fine.
Any idea what could be the cause of this failure to crawl? I have pasted a few examples from the report
|
500 : TimeoutError
http://everythingforthegirl.co.uk/index.php/accessories.html 500 1 0 500 : Error
http://everythingforthegirl.co.uk/index.php/accessories/bags.html 500 1 0 500 : Error
http://everythingforthegirl.co.uk/index.php/accessories/gloves.html 500 1 0 500 : Error
http://everythingforthegirl.co.uk/index.php/accessories/purses.html 500 1 0 500 : TimeoutError
http://everythingforthegirl.co.uk/index.php/accessories/sunglasses.html | 500 | 1 | 0 |
Am extra puzzled why the messages say time out. The server dedicated is 8 core with 32 gb of ram, the pages ping for me in about 1.2 seconds. What is the rogerbot crawler timeout?
Many thanks
Carl
-
You're absolutely correct, hopefully this answered your question!
-
Thanks, will check out that plugin. So, in other words, the pages are loading fine for the user but sending out an error to the bots instead of the loaded ok message. That doesn't sound good!!
On the plus side, at least it has stopped Roger noticing some of the pages have up to 600 links on them because of all the retailer and manufacturer filtering options!!
Many thanks, Carl
-
Hi Carl,
You're a lucky man (sarcastic), your paging are loading just normally but are indeed giving the wrong status code: 500 for me. This is probably caused by some of the settings in Magento or you server as the normal status code for working pages should be a 200 OK.
That's probably also why Rogerbot didn't timeout on the pages but got a 500 while the pages were working. Good luck fixxing this!
Btw, I highly recommend using the Redirect Plugin for Chrome by Ayima.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to overcome Connection Timeout Status Error?
My website contains 110+ pages in which 70 are CONNECTION TIMEOUT while checking in Screaming Frog. Can someone help me in getting this solved? My website Home Page Sanctum Consulting.
Moz Pro | | Manifeat90 -
Huge spike in crawl errors today - mozbot ignoring noindex tag?
Hi Mozzers, Today I received a ton of errors and warnings in my weekly crawl due to the mozbot crawling my noindex'd search results pages, such as this - http://www.consumerbase.com/Mailing-Lists.html?q=Construction&type=bus&channel=all&page=7&order=title&orderBy=DESC See image: http://screencast.com/t/qaZzq78j2Udx Anyone else seen a similar error this week with their crawl? Thanks!
Moz Pro | | Travis-W0 -
"Loading" error in Open Site Explorer
The "top pages" keeps hanging while trying to retrieve social metrics "FB Shares" "Tweets" and "Google +1". I've tried in Chrome and Firefox. Can I just turn these off? ose-hanging.jpg
Moz Pro | | cpaddock0 -
Why don't Google+ URL's work in OSE?
Is there any reason why Google+ URLs does not work in OSE? Is it just that it is a secure URL or is there something bigger there? Why? Be cool to determine every website the person has been published on; especially if it is rel="author" verified. Jeff
Moz Pro | | WebBizIdeas1 -
Why do I keep getting "more than one canonical URL tag" on-page factor when, in fact, there is always only one?
The following are pages that SEOMOZ says have "more than one canonical URL tag" but they all have only one. Can someone help me understand this?http://www.lasercenterny.com/Laser-Hair-Removal-Binghamton/tabid/1950/Default.aspxhttp://www.lasercenterny.com/Hair-Removal-Binghamton-NY/tabid/1949/Default.aspxhttp://www.lasercenterny.com/Hair-Removal-Binghamton/tabid/1948/Default.aspx
Moz Pro | | SmartWebPros0 -
Wordpress Plugin Causing Mobile Switcher 404 Errors
Hi All, Has anyone seen this 404 error before? I installed a wordpress plugin that would allow users to switch between a mobile and desktop theme. This was about 6 months ago and I thought nothing of it. Since I am new to SeoMoz, I have become aware of this lovely problem. After setting up my campaign, I now have 1600 something 404 errors due to this plugin. It looks like this plugin creates 4 to 5 links for each one of my posts and they all return up as a 404 error. Example: http://frogfanreport.com/football/page/36/ or http://frogfanreport.com/football/page/36/?wpmp_switcher=desktop I just noticed this morning in Google Webmaster tools that the errors are starting to show up there. Has anyone seen this? Or know if this is a problem and what to do? zach
Moz Pro | | TCUFrogFanReport0 -
It won't let me print the secon or third pages of site errors.
When I place the site errors page in pdf format it won't let me print the second or any of the other webpages containing content about my site. Does any one know why?
Moz Pro | | ibex0 -
4xx (not found) errors seem spurious, caused by a "\" added to the URL
Hi SEOmoz folks We're getting a lot of 404 (not found) errors in our weekly crawl. However the weird thing is that the URLs in question all have the same issue. They are all a valid URL with a backsalsh ("") added. In URL encoding, this is an extra %5C at the end of the URL. Even weirder, we do not have any such URLs in our (Wordpress-based) website. Any insight on how to get rid of this issue? Thanks
Moz Pro | | GPN0