What's the best way to eliminate "429 : Received HTTP status 429" errors?
-
My company website is built on WordPress. It receives very few crawl errors, but it do regularly receive a few (typically 1-2 per crawl) "429 : Received HTTP status 429" errors through Moz.
Based on my research, my understand is that my server is essentially telling Moz to cool it with the requests. That means it could be doing the same for search engines' bots and even visitors, right? This creates two questions for me, which I would greatly appreciate your help with:
-
Are "429 : Received HTTP status 429" errors harmful for my SEO? I imagine the answer is "yes" because Moz flags them as high priority issues in my crawl report.
-
What can I do to eliminate "429 : Received HTTP status 429" errors?
Any insight you can offer is greatly appreciated!
Thanks,
Ryan -
-
I have a customer that is using GoDaddy website hosting (at least according to BuiltWith) and I'm experiencing this same issue.
Any updates on this experiment from user rsigg? I'd love to know if I can remove this from my customer's robots file...
FWIW, Netrepid is a hosting provider for colocation, infrastructure and applications (website hosting being considered an application) and we would never force a crawl delay on a Wordpress install!
Not hating on the other hosting service providers... #justsayin
-
I am also on the same hosting and they have not been able to help with the 429. I have now started getting 429 errors when I attempt to login. Definitely something wrong with wp premium hosting.
-
Interesting. I look forward to hearing your results, as my robots.txt file is also set to:
Crawl-delay: 1.
-
We host on Media Temple's Premium WordPress hosting (which I do not recommend, but that's another post for another place), and the techs there told me that it could be an issue with the robots.txt file:
"The issue may be with the settings in the robots.txt file. It looks fine to me but the "Crawl-delay" line might be causing issues. I understand. For the most part, crawlers tend to use robots.txt to determine how to crawl your site, so you may want to see if Moz requires some special settings in there to work correctly."
Ours is set to:
Crawl-delay: 1
I haven't tried changing these values yet in our file, but may experiment with this very soon. If I get results, I'll post back here as well as start a new forum thread.
-
Chase,
They ran a bunch of internal diagnostic tools on my site, and were unable to replicate the 429 errors. They ended up telling me exactly what they told you. I haven't noticed any issues with my site's rankings, or any flags in Webmaster Tools, so it looks like they are right so far. I just hate logging into Moz and seeing all those crawl errors!
-
What'd they say Ryan? Having the same issue and just contacted Godaddy who told me that basically Moz's software is pinging my client's server too frequently so Godaddy is temporarily blocking their IP. They said it's not a concern though as they would never block Google from pinging/indexing the site.
-
Many thanks - I will contact them now!
-
Contact your host and ask let them know about the errors. More than likely they have mod_sec enabled to limit request rates. Ask them to up the limit that you are getting 429 errors from crawlers and you do not want them.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Best Way to Seemlessly Use Moz and Google Analytics
Hi All, I have Moz Pro and I am beginning to explore Moz Analytics. I currently use Google Analytics for pretty much everything and I am almost at expert level at using the GA interface so I'm ready to learn a new tool and have a second source of data! My question is: what's the best way to get the most of Moz Analytics and GA congruently? Off hand I know that Moz's keyword, rank, and link data, including competitive insights, is more comprehensive than GA. But how else can I use Moz and get the most out of the Moz Analytics? Looking to really dive in and discuss how we can make use of both! Thanks - Vee
Moz Pro | | Symmetri0 -
What's the best way to keep track of keyword rankings
Here's the deal. I keep track of my keyword rankings with the help of Rank Tracker from seopowersuite.com for the most part. I ran it on a daily basis and my keyword was not in top 100 for a few months. Moz.com panel shows pretty much the same (not in top 50) for the same months. That said, if I check that keyword ranking in my Google Webmaster Tools (avg. position) it says that its position (ranking?) was on average: 49, 7, 7, 8 (for the last four months). So, I'm not sure how it's even possible? How come Rank Tracker and Moz don't see any rankings and Google gives me sorta decent avg. positions at the same time. I assume that avg. postion means that same as avg. ranking, right? I'm not sure what I'm missing here.
Moz Pro | | VinceWicks0 -
What should I put in 'Define Branded Keyword Rules' -Starting a Campaign
Hello, I am a new user here (this seems really interesting!), but english level is not very good (I am spanish) and I dont understand what means 'Define Branded Keyword Rules' Hope someone can explain me it in easy words so I can understand Thank you very much! 1362443047.jpeg
Moz Pro | | matiw0 -
Duplicate content error?
I am getting a duplicate content error for the following pages: http://www.bluelinkerp.com/products/accounting/index.asp http://www.bluelinkerp.com/products/accounting/ But, of course, the 2nd link is just an automatic redirect to the index file, is it not? Why is it thinking it is a different URL? See image. NJfxA.png
Moz Pro | | BlueLinkERP0 -
Best Keyword Difficulty Tool?
Hi All, I'm a bit frustrated with the fact that I can only enter 5 words at a time into SEOmoz's keyword difficulty tool. Does anyone know of a better way (or tool) to analyze keyword difficulty for hundreds of keywords?
Moz Pro | | nicole.healthline0 -
Whats the best way to research website relevant keywords
I wanted to know the general practices SEO marketers use when initially researching and revisiting relevant keywords for a website. I use this process Brain storm a list of keywords, between me and the client Check out competitors websites & SERP's look at Google adwords traffic analyser ( mostly as a point of reference and for increased ideas) How do others do it? what process do you find works - i'm based in the Uk would be great to get a UK perspective. Obviously you can't optimise for all keywords in your list how best is it to decide on the best value or less competitive keywords? I struggle to get stats for local keywords( not enough search data) - most my clients are local businesses and have limited surrounding service areas. thanks
Moz Pro | | Bristolweb0 -
Significance of SEOmoz`s keyword difficulty
Hello everybody, currently, I am working on a project that will be available in multiple languages. My problem concerns the difference of the German and English language in relation to the SEOmoz`s Keyword difficulty (KWD). I just finished my German keyword research and got a satisfying collection of keywords I can use according to the KWD, page/ domain authority, Backlinks and Onpage-check. Now, as I am doing my research for the English market I find it hard to find any appropriate keywords with a satisfying KWD. I imagined the ranking on the English market to be much harder but now I doubt to ever rank for any useful, effective keyword on my list, because most of them are over 70 (some important over 80). According to the research it seems to have something to do with the authority strength of the pages, trust, etc, because they do not have more backlinks. Since they are query independent it is very hard for me to get a feeling for those metrics (I am new at SEO). So here are my questions: Due to your experiences, is it even possible to derank such pages or should I simply save the effort? How significant is the SEOmoz`s KWD? Would you always do Onpage optimization (text, internal linking) in case you earn more trust, authority, etc in the future, just to have your site optimized on the best keyword possible, even though you know that you cannot rank for that keyword (Cue: User-experience)? Or would you switch to less significant words (e.g. flashcards -> index cards) where you have a chance to rank but which is not that effective with traffic? Or focus on longtail, if possible? Many thanks
Moz Pro | | cramlr0 -
How do we use SEOmoz to track Local Searches? Is there a way to set the location from which the campaign tracker is "searching"?
It seems that there is no way to set a parameter for location. In Places, I'm able to define my targeted region. How does SEOmoz mimic that localization? Thank for any help! -- Chris
Moz Pro | | ChrisPalle1