Unsolved The Moz.com bot is overloading my server
-
. How to solve it?
-
For a step-by-step guide on setting up the tool, check out the solara executor download tutorial.
-
maybe crawl delay will help.
-
@paulavervo
Hi,
We do! The best way to chat with us is via our contact form or direct email. We also have chat within Moz Pro.
Please contact us via help@moz.com or https://moz.com/help/contact
We will be happy to help.
Cheers,
Kerry. -
very nice brother i like it very good keep it up !
-
very nice !
-
does the moz team even monitor this forum?
-
If the Moz.com bot is overloading your server, there are several steps you can take to manage and mitigate the issue effectively. First, you can adjust the crawl rate in your
robots.txt
file by specifying a crawl delay for the Moz bot using directives likeUser-agent: rogerbot
andUser-agent: dotbot
, followed byCrawl-delay: 10
to make the bot wait 10 seconds between requests. If this does not suffice, you can temporarily block the bot by disallowing it in yourrobots.txt
file. Additionally, it's a good idea to contact Moz’s support team to explain the issue, as they may offer solutions to adjust the crawl rate for your site. Implementing server-side rate limiting is another effective strategy. For Apache servers, you can add rules in your.htaccess
file to return a 429 Too Many Requests status code to the Moz bots, while for Nginx servers, you can set up rate limiting in your configuration file to control the number of requests per second from a single user or IP address. Monitoring your server’s performance and log files can help identify specific patterns or peak times, allowing you to fine-tune your settings. Furthermore, using a Content Delivery Network (CDN) can help distribute the load by caching content and serving it from multiple locations, reducing the direct impact on your server caused by crawlers. By taking these steps, you can manage the load from the Moz.com bot and maintain your server’s stability and responsiveness.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved MOZ Crawler Stalled
MOZ crawl has been waiting since April 2nd. Whats the deal? I am not able to "recrawl", so i have been stuck in limbo.
Product Support | | WebMarkets0 -
Moz crawler is not able to crawl my website
Hi, i need help regarding Moz Can't Crawl Your Site i also share screenshot that Moz was unable to crawl your site on Mar 26, 2022. Our crawler was not able to access the robots.txt file on your site. This often occurs because of a server error from the robots.txt. Although this may have been caused by a temporary outage, we recommend making sure your robots.txt file is accessible and that your network and server are working correctly. Typically errors like this should be investigated and fixed by the site webmaster.
Technical SEO | | JasonTorney
my robts.txt also ok i checked it
Here is my website https://whiskcreative.com.au
just check it please as soon as possibe0 -
Should I set blog category/tag pages as "noindex"? If so, how do I prevent "meta noindex" Moz crawl errors for those pages?
From what I can tell, SEO experts recommend setting blog category and tag pages (ie. "http://site.com/blog/tag/some-product") as "noindex, follow" in order to keep the page quality of indexable pages high. However, I just received a slew of critical crawl warnings from Moz for having these pages set to "noindex." Should the pages be indexed? If not, why am I receiving critical crawl warnings from Moz and how do I prevent this?
Moz Pro | | NichGunn0 -
Why doesn't Moz crawl whole pages of our website to report All On-Page issues?
Hi friends & mozzers, How can't Moz crawl whole pages of our website: https://www.4atvtires.com/ to report All Serious On-Page issues. We have more than 15000 product pages. And how could it be possible that Moz isn't able to crawl whole, just got crawl report upto 258 pages of our website, and also I can experience the same in Google webmaster ?? Please help to fix this issue as early as possible. Regards,
Moz Pro | | BigSlate
Rann0 -
Following moz guideliness complety, will it certainly improve keyword rankings, domain authority
Hi! G’day to you. I am Naresh from Dotline Infotech. I’d like to take up couple of concerns to be clarified for my better understanding on Moz tool. Indeed, Moz is a nice SEO tool that helps in improving Search engine optimization comprehensive process. We are moz pro member since a year. Moz has equally bifurcated every SEO activity and rate it according to its own benchmark. I intend to ask a series of concerns; Q1. Moz has page grading system A to F. According to Moz guidelines, if the keywords on the page gets repeated minimum 4 times, Moz consider it as ‘A’ grading (one of Moz metric to evaluating the page). Whereas, Google indicates repetition of same keywords time and again is treated as keyword stuffing. According to me, if keywords are already within the content one or two time and furnishing relevant information, why keywords re-occurrence to be 4 time. Does the repetition of same keywords 4 times will lead to give more valuable information? Just a request to let me share why we should unnecessarily repeat keywords? Q2. As per Google guidelines, only 3 keywords can be placed in title tag and also advisable to use only one h1 tag instead of multiple. Among 3 keywords in title tag, only one main keyword can be used in H1 tag. Moz gives much weight age to that particular keyword placed in H1 tag but rest two keywords does not get any importance. I just like to know if we use three H1 tags within page, will it increase the page grading or not. Q3. Does Moz tool have its own index data or refer to Google’s ones? Q4. Following the Moz guidelines completely, will it be 100% sure all our keyword rankings, page authority, domain authority certainly be improved. Looking forward to hear you soon. Thanks
Moz Pro | | dotlineseo0 -
How Moz takes a page title is duplicate?
Suppose i have added suffix and prefix to each of my product (ex: i have two tittles like buy online t-shirt at abc.com & buy online poster at abc.com, so in this buy online and abc.com are suffix and prefix) so .. will it take these two page tittles as duplicates ?
Moz Pro | | vayush0 -
Moz bar down?
Is the Mozbar not working for link data? Whats up with moz I signed up for a free trial and nothing works....
Moz Pro | | JohnWalker0 -
Seo Moz for Clients
One thing I recently pondered following the introduction of logos on reports bringing that level of personalisation for the customers is ... would it be somehow possible to take it a step further and have a login on your own personal site that could somehow enable clients to download and look at the information as and when they wanted. An almost mini sep moz for clients where customers login through your own site ?
Moz Pro | | onlinemediadirect0