Unsolved The Moz.com bot is overloading my server

[paulavervo]

. How to solve it?

[ChotiGolpo]

maybe crawl delay will help.

[KerryDuggan]

@paulavervo
Hi,
We do! The best way to chat with us is via our contact form or direct email. We also have chat within Moz Pro.
Please contact us via help@moz.com or https://moz.com/help/contact
We will be happy to help.
Cheers,
Kerry.

[Abdul9865]

very nice brother i like it very good keep it up !

[Abdul9865]

very nice !

[paulavervo]

does the moz team even monitor this forum?

[Alex2211]

If the Moz.com bot is overloading your server, there are several steps you can take to manage and mitigate the issue effectively. First, you can adjust the crawl rate in your robots.txt file by specifying a crawl delay for the Moz bot using directives like User-agent: rogerbot and User-agent: dotbot, followed by Crawl-delay: 10 to make the bot wait 10 seconds between requests. If this does not suffice, you can temporarily block the bot by disallowing it in your robots.txt file. Additionally, it's a good idea to contact Moz’s support team to explain the issue, as they may offer solutions to adjust the crawl rate for your site. Implementing server-side rate limiting is another effective strategy. For Apache servers, you can add rules in your .htaccess file to return a 429 Too Many Requests status code to the Moz bots, while for Nginx servers, you can set up rate limiting in your configuration file to control the number of requests per second from a single user or IP address. Monitoring your server’s performance and log files can help identify specific patterns or peak times, allowing you to fine-tune your settings. Furthermore, using a Content Delivery Network (CDN) can help distribute the load by caching content and serving it from multiple locations, reducing the direct impact on your server caused by crawlers. By taking these steps, you can manage the load from the Moz.com bot and maintain your server’s stability and responsiveness.