What's the best way to eliminate "429 : Received HTTP status 429" errors?
-
My company website is built on WordPress. It receives very few crawl errors, but it do regularly receive a few (typically 1-2 per crawl) "429 : Received HTTP status 429" errors through Moz.
Based on my research, my understand is that my server is essentially telling Moz to cool it with the requests. That means it could be doing the same for search engines' bots and even visitors, right? This creates two questions for me, which I would greatly appreciate your help with:
-
Are "429 : Received HTTP status 429" errors harmful for my SEO? I imagine the answer is "yes" because Moz flags them as high priority issues in my crawl report.
-
What can I do to eliminate "429 : Received HTTP status 429" errors?
Any insight you can offer is greatly appreciated!
Thanks,
Ryan -
-
I have a customer that is using GoDaddy website hosting (at least according to BuiltWith) and I'm experiencing this same issue.
Any updates on this experiment from user rsigg? I'd love to know if I can remove this from my customer's robots file...
FWIW, Netrepid is a hosting provider for colocation, infrastructure and applications (website hosting being considered an application) and we would never force a crawl delay on a Wordpress install!
Not hating on the other hosting service providers... #justsayin
-
I am also on the same hosting and they have not been able to help with the 429. I have now started getting 429 errors when I attempt to login. Definitely something wrong with wp premium hosting.
-
Interesting. I look forward to hearing your results, as my robots.txt file is also set to:
Crawl-delay: 1.
-
We host on Media Temple's Premium WordPress hosting (which I do not recommend, but that's another post for another place), and the techs there told me that it could be an issue with the robots.txt file:
"The issue may be with the settings in the robots.txt file. It looks fine to me but the "Crawl-delay" line might be causing issues. I understand. For the most part, crawlers tend to use robots.txt to determine how to crawl your site, so you may want to see if Moz requires some special settings in there to work correctly."
Ours is set to:
Crawl-delay: 1
I haven't tried changing these values yet in our file, but may experiment with this very soon. If I get results, I'll post back here as well as start a new forum thread.
-
Chase,
They ran a bunch of internal diagnostic tools on my site, and were unable to replicate the 429 errors. They ended up telling me exactly what they told you. I haven't noticed any issues with my site's rankings, or any flags in Webmaster Tools, so it looks like they are right so far. I just hate logging into Moz and seeing all those crawl errors!
-
What'd they say Ryan? Having the same issue and just contacted Godaddy who told me that basically Moz's software is pinging my client's server too frequently so Godaddy is temporarily blocking their IP. They said it's not a concern though as they would never block Google from pinging/indexing the site.
-
Many thanks - I will contact them now!
-
Contact your host and ask let them know about the errors. More than likely they have mod_sec enabled to limit request rates. Ask them to up the limit that you are getting 429 errors from crawlers and you do not want them.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Hi guys What the best way to adress duplicate content on photo gallery?
inside my moz report for duplicate contentit says that the photo gallery has duplicate content. let me post and example. is saying this site->http://www.yoursite.com//photogallery/name-of-the-page site photogallery category page name its being duplicated to all these other urls : http://www.yoursite.com//photogallery/name-of-the-page-categoryone http://www.yoursite.com//photogallery/name-of-the-page-categorytwo http://www.yoursite.com//photogallery/name-of-the-page-categorythree http://www.yoursite.com//photogallery/name-of-the-page-categoryfour and so on! each one has it own canonical tag to its own individual page. the site structure is this: http://www.yoursite.com//photogallery/ in here there are all the links pointing to the right categorypage ie: http://www.yoursite.com//photogallery/ >>>> http://www.yoursite.com//photogallery/categoryone pic 1 pic 2 pic 3 http://www.yoursite.com//photogallery/categorytwo pic 1 pic 2 pic 3 http://www.yoursite.com//photogallery/categorythree pic 1 pic 2 pic 3 http://www.yoursite.com//photogallery/categoryfour pic 1 pic 2 pic 3 So i don't know how to interpret Moz diagnose. how could i interpret moz reports to find out what to fix and how to fix it? Sorry for the long post! ;
Moz Pro | | surgeonsadvisor0 -
Mozscape index dosen't update
Last Mozscape index update: July 11, 2013. Next Mozscape index update: August26, 2013 Today is August26.Why mozscape doesnt updated?
Moz Pro | | vahidafshari451 -
Where To Get a *Hint of My Beta Invite Status?
Wondering if I'm missing a thread somewhere with updates on new analytics dashboard (beta) invites? I'm not surprised I'm not first in line, but be nice to know if it's 2013...or 14?...or 15? 🙂
Moz Pro | | InsightRiot300 -
Error in Moz duplicate content reports
Hi - I've run the Moz campaign on a client's site. Moz is saying that there are duplicate content errors, and when I look at the errors it is showing that they are all to do with the non-www URLs having being duplicated in the www form of the URLs. However this is not the case - all the non-www URLs are all 301 redirected to the www URLs. Is this an error in the Moz tool? Has anybody experienced something similar?
Moz Pro | | rorynatkiel0 -
Is there any way to move keywords from one Campaign to another?
We recently moved content off a subdomain onto our main www subdomain. Each of these was previously tracked as its own campaign. Now that the content is consolidated, I'd like to move the keywords that we were tracking on the first campaign over to the second. I don't see an option to migrate, or export/import keywords. Is there any (non-manual) way to do this? Thanks
Moz Pro | | doxo1 -
Best kw difficulty score to go for as a newbie
I have been using seomoz keyword difficulty tool for quite a while. My question is, which is the best keyword difficulty score to go for as a newbie? I have seen many keywords at the range of 40 to 50 but I dont know if it will be easier to rank them esp with a new website.
Moz Pro | | dronehawk0 -
Current on-page best practices
Given all the recent talk about over optimization, when was the last time SEOMoz updated the on-page report card tool? Rand wrote an excellent piece on Perfect On-Page Optimization (which is great, and thanks) in summer 2009. Is that still best practice 3 years later (and post-Penguin/Panda)? If not, has the SEOMoz on-page report card tool been updated to reflect current thinking for on-page best-practices? I know the higher level concept is "write for humans, not for bots" but if you can do both (and not create an unreadable seo-frankenpage) then why not? Does getting an "A" grade wreak of over optimization now? Should I use the key phrase at the start of the title, h1, and strong (or bold) elements on a page? Should have an image with file name and alt text equal to (or containing at the start) the key phrase?
Moz Pro | | scanlin2 -
What's name of SEOmoz and Open Site Explorer robots?!
I would like to exclude in robots.txt SEOmoz and Open Site Explorer bots to don't let them index my sites… what's their names?
Moz Pro | | cezarylech0