What's the best way to eliminate "429 : Received HTTP status 429" errors?
-
My company website is built on WordPress. It receives very few crawl errors, but it do regularly receive a few (typically 1-2 per crawl) "429 : Received HTTP status 429" errors through Moz.
Based on my research, my understand is that my server is essentially telling Moz to cool it with the requests. That means it could be doing the same for search engines' bots and even visitors, right? This creates two questions for me, which I would greatly appreciate your help with:
-
Are "429 : Received HTTP status 429" errors harmful for my SEO? I imagine the answer is "yes" because Moz flags them as high priority issues in my crawl report.
-
What can I do to eliminate "429 : Received HTTP status 429" errors?
Any insight you can offer is greatly appreciated!
Thanks,
Ryan -
-
I have a customer that is using GoDaddy website hosting (at least according to BuiltWith) and I'm experiencing this same issue.
Any updates on this experiment from user rsigg? I'd love to know if I can remove this from my customer's robots file...
FWIW, Netrepid is a hosting provider for colocation, infrastructure and applications (website hosting being considered an application) and we would never force a crawl delay on a Wordpress install!
Not hating on the other hosting service providers... #justsayin
-
I am also on the same hosting and they have not been able to help with the 429. I have now started getting 429 errors when I attempt to login. Definitely something wrong with wp premium hosting.
-
Interesting. I look forward to hearing your results, as my robots.txt file is also set to:
Crawl-delay: 1.
-
We host on Media Temple's Premium WordPress hosting (which I do not recommend, but that's another post for another place), and the techs there told me that it could be an issue with the robots.txt file:
"The issue may be with the settings in the robots.txt file. It looks fine to me but the "Crawl-delay" line might be causing issues. I understand. For the most part, crawlers tend to use robots.txt to determine how to crawl your site, so you may want to see if Moz requires some special settings in there to work correctly."
Ours is set to:
Crawl-delay: 1
I haven't tried changing these values yet in our file, but may experiment with this very soon. If I get results, I'll post back here as well as start a new forum thread.
-
Chase,
They ran a bunch of internal diagnostic tools on my site, and were unable to replicate the 429 errors. They ended up telling me exactly what they told you. I haven't noticed any issues with my site's rankings, or any flags in Webmaster Tools, so it looks like they are right so far. I just hate logging into Moz and seeing all those crawl errors!
-
What'd they say Ryan? Having the same issue and just contacted Godaddy who told me that basically Moz's software is pinging my client's server too frequently so Godaddy is temporarily blocking their IP. They said it's not a concern though as they would never block Google from pinging/indexing the site.
-
Many thanks - I will contact them now!
-
Contact your host and ask let them know about the errors. More than likely they have mod_sec enabled to limit request rates. Ask them to up the limit that you are getting 429 errors from crawlers and you do not want them.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Duplicate content flagged by Moz that's not actually duplicate content at all
Hi, Moz has flagged a whole lot of pages as dupe content, but I cannot see how they qualify as such.
Moz Pro | | Caro-O
Not sure if I'm allowed to post actual URLs here....happy to if I can, but I feel certain that the pages are not 90% similar. Has anyone else had this experience? ~Caro1 -
I have a new website and would like to use the keywords my competitors are using and was wondering the best way to take a look using Moz Pro?
The website is www.fapwrap.com - it's a new adult wall decals website. [editor's note: may not be safe for some workplaces] The wall decals we have are obviously a bit more niche than the bigger competitors, but I want to get into the first page to see how it helps us. I'm slowly taking over some kws like "xbox wraps" and "ps4 wraps" but those have low search volume and I feel like "wall decals" will help - though it's highly competitive. Suggestions? I just kinda want to plug in my competition and pick from their list of existing kws..... Ha.
Moz Pro | | JohnnyRoq0 -
Good job! This URL received an grade A ?
What does this mean ? This page still ranks very bad at Google. So what does it mean that it recives a grade A ?Also, if this URL recives Grade A. It should clearly be optmizied very good on the Keyword. Stil its on page 9 in google. That is very very low on this keyword. Alot of bad blogs, foreign pages, pages without the keyword in heading, pages without any good content etc. rank better.Does this score have anything to do with ranking on google ?Something is clearly wrong with this page. The on-grade tool wont tell me what that it is. And probably dont understand either, since it gives A. Is there anywhere I can check this page on what is wrong with it ?http://www.butikksiden.no/archives/5-toffe-canada-goose-jakker and Canada Goose jakker Re-Grade Pa Good Job! This URL received an A grade
Moz Pro | | butikksiden0 -
What's the best way to keep track of keyword rankings
Here's the deal. I keep track of my keyword rankings with the help of Rank Tracker from seopowersuite.com for the most part. I ran it on a daily basis and my keyword was not in top 100 for a few months. Moz.com panel shows pretty much the same (not in top 50) for the same months. That said, if I check that keyword ranking in my Google Webmaster Tools (avg. position) it says that its position (ranking?) was on average: 49, 7, 7, 8 (for the last four months). So, I'm not sure how it's even possible? How come Rank Tracker and Moz don't see any rankings and Google gives me sorta decent avg. positions at the same time. I assume that avg. postion means that same as avg. ranking, right? I'm not sure what I'm missing here.
Moz Pro | | VinceWicks0 -
Quote "There is no SEO software that can help optimize a site"
Hi all I am a developer for a small agency in the UK. I am using SEOmoz for a couple of our clients and I am also using it to learn from. So far I am finding it a great tool and this forum is also incredibly helpful. Now I also visit another forum called HighRankings.com and this months newsletter is a list of points that are defined as "SEO Mistakes". In it, point number 2 reads: "Repeat after me: There is no specific number of times a keyword phrase should be used in my content. There is no magic number of words that my pages should have written on them. And there is no best number of words or phrases that belong in a Title tag. And most of all: There is no SEO software that can optimize my website (despite the claims of their creators). Use your common sense to optimize your site! Learn how your target audience searches for products, services and information such as yours, and write about it accordingly on your website. Then write to make an emotional connection with your visitors so that they'll convert into happy customers." Now as a user of SEOmoz I wondered what others thought of this comment? Many thanks for reading. Kris
Moz Pro | | yousayjump0 -
Is there any way to change the domain in a campaign here?
When I started the campaign I used site.com but now I have an external blog at blog.site.com and is hosted on tumblr. This site gets lots of errors showing on the campaign because insists on following no-follows. If I change the campaign to www.site.com then blog.site.com will not be part of the campaign. It does say in the help that it will NOT track sub-domains, but it does and hence my problem. So, I am not sure if the rest of the info there is old. 'Does the Web App track the subdomains of my campaign’s domain?
Moz Pro | | oznappies
A. Unfortunately we do not currently track subdomains as a part of the
domain you enter. Instead, you must create another campaign slot for any
subdomains you wish to track.' Any ideas on where or how to do this?0 -
Confounding "Accessible to Engines" error?
Most of the pages on our site "Accessible to Engines" test in the SEOmoz reports. We cannot find any problem with the code and it's largely identical to the few pages that come up with an "A" score. One item that may be a reason is that we use meta http-equiv="refresh" content="600; For example in www.weatherzone.com.au/nsw/sydney/sydney We use this to fresh dynamic content on our site. Do search engines penalise pages that use this form of page refresh? Alternatively, is there a known bug in the SEOmoz "Accessible to Engines" report? Many thanks
Moz Pro | | weatherzone0 -
Where in SEOMoz pro can I determine how many .edu and .gov links a competitor's site has?
Like in the SEOBook toolbar where you can click the information button and get the number of links to the domain and specific page for .edu and .gov sites. Thanks, Daniel.
Moz Pro | | iSenseWebSolutions0