What's the best way to eliminate "429 : Received HTTP status 429" errors?
-
My company website is built on WordPress. It receives very few crawl errors, but it do regularly receive a few (typically 1-2 per crawl) "429 : Received HTTP status 429" errors through Moz.
Based on my research, my understand is that my server is essentially telling Moz to cool it with the requests. That means it could be doing the same for search engines' bots and even visitors, right? This creates two questions for me, which I would greatly appreciate your help with:
-
Are "429 : Received HTTP status 429" errors harmful for my SEO? I imagine the answer is "yes" because Moz flags them as high priority issues in my crawl report.
-
What can I do to eliminate "429 : Received HTTP status 429" errors?
Any insight you can offer is greatly appreciated!
Thanks,
Ryan -
-
I have a customer that is using GoDaddy website hosting (at least according to BuiltWith) and I'm experiencing this same issue.
Any updates on this experiment from user rsigg? I'd love to know if I can remove this from my customer's robots file...
FWIW, Netrepid is a hosting provider for colocation, infrastructure and applications (website hosting being considered an application) and we would never force a crawl delay on a Wordpress install!
Not hating on the other hosting service providers... #justsayin
-
I am also on the same hosting and they have not been able to help with the 429. I have now started getting 429 errors when I attempt to login. Definitely something wrong with wp premium hosting.
-
Interesting. I look forward to hearing your results, as my robots.txt file is also set to:
Crawl-delay: 1.
-
We host on Media Temple's Premium WordPress hosting (which I do not recommend, but that's another post for another place), and the techs there told me that it could be an issue with the robots.txt file:
"The issue may be with the settings in the robots.txt file. It looks fine to me but the "Crawl-delay" line might be causing issues. I understand. For the most part, crawlers tend to use robots.txt to determine how to crawl your site, so you may want to see if Moz requires some special settings in there to work correctly."
Ours is set to:
Crawl-delay: 1
I haven't tried changing these values yet in our file, but may experiment with this very soon. If I get results, I'll post back here as well as start a new forum thread.
-
Chase,
They ran a bunch of internal diagnostic tools on my site, and were unable to replicate the 429 errors. They ended up telling me exactly what they told you. I haven't noticed any issues with my site's rankings, or any flags in Webmaster Tools, so it looks like they are right so far. I just hate logging into Moz and seeing all those crawl errors!
-
What'd they say Ryan? Having the same issue and just contacted Godaddy who told me that basically Moz's software is pinging my client's server too frequently so Godaddy is temporarily blocking their IP. They said it's not a concern though as they would never block Google from pinging/indexing the site.
-
Many thanks - I will contact them now!
-
Contact your host and ask let them know about the errors. More than likely they have mod_sec enabled to limit request rates. Ask them to up the limit that you are getting 429 errors from crawlers and you do not want them.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why can't I see last week's stats?
Can see stats week ending May 1st but nothing after that- and it's May 9th!?
Moz Pro | | locumhunter0 -
Pro newbie - What is a "campaign"?
I signed up three weeks ago and the first email to me begins with: The first step in PRO is to set up your campaign. Once we start tracking your site and social media accounts, you'll start receiving handy reports of your data. You can also surface site issues right in the app to help you prioritize which fixes you can make for immediate results. At this point I don't know what kind of "campaign" this is or what the "app" is. I have also lodged a ticket because something seems to be broken about my Pro status. I can obviously participate here, so I have a Pro status, but at OSE, it says "Social metrics only available to paid Moz subscribers. Learn more" and hit my Advanced Report limit for the day without seeing any such report.
Moz Pro | | trainSEM0 -
What's the best tool to use to compare competirors
A client of ours has asked us to compare their search rankings to competitors. What's the best tool to use in SEOMoz to do this?
Moz Pro | | BillyBobGriffin0 -
Way too many 301 redirects
Hi Guys Posting here again in regard to 301 redirects, SEOMOZ has just finished my second crawl, and its showing me nearly 4k 301 redirects, i am only tracking my new site which by the way changed domain name , however all im doing is tracking the new domain and there are no 301 implemented on that side (Except Canonical), so was wondering how SEOMoz is finding 4k of redirects. do you think it could be chaining somewhere ? as all the tests look good or could there be a problem with the reporting
Moz Pro | | kellymandingo0 -
Optimization report card says "F" but I'm ranking #1
Using the SEOMoz pro tools and several of the terms I have listed get a report card of "F" but they also show Google rank of #1, #2. Is this unusual? And should I spend a lot of time "fixing" this if we're already ranking high?
Moz Pro | | schof0 -
Using the 'Organic Traffic Data ' feature.
I don't have access to the clients Google Analytics Password. Can you set it up without being redirected to Google webmasters? Is there a way to insert a simple code into the website and then have SEOmoz double check the code? Thanks so much
Moz Pro | | Robin_Jennings0 -
SEOmoz bot and "noindex"
As a recent newbie to SEOmoz, I've been implementing some suggestions and doing a general tidy up. I removed URL's from our robots txt, and rolled out instead the noindex meta tag to pages we don't want indexed. But surprised to see issues that are now flagged from the last crawl by the moz bot from pages that have this meta tag? Does the SEOmoz bot not ignore this tag? Just want to make sure I've implemented it correctly, so the google bot does ignore it. Meta tag syntax is and is placed below the title tag. cheers Steve
Moz Pro | | sjr4x40 -
Confounding "Accessible to Engines" error?
Most of the pages on our site "Accessible to Engines" test in the SEOmoz reports. We cannot find any problem with the code and it's largely identical to the few pages that come up with an "A" score. One item that may be a reason is that we use meta http-equiv="refresh" content="600; For example in www.weatherzone.com.au/nsw/sydney/sydney We use this to fresh dynamic content on our site. Do search engines penalise pages that use this form of page refresh? Alternatively, is there a known bug in the SEOmoz "Accessible to Engines" report? Many thanks
Moz Pro | | weatherzone0