Hi,
I will double-check with firewall settings in our servers. Could you please share moz-pro site-crawler roger bot IP addresses/range? We will verify against our firewall rules.
Thanks
Shashi
Welcome to the Q&A Forum
Browse the forum for helpful insights and fresh discussions about all things SEO.
Hi,
I will double-check with firewall settings in our servers. Could you please share moz-pro site-crawler roger bot IP addresses/range? We will verify against our firewall rules.
Thanks
Shashi
I am looking for roger bot site crawler IP addresses Please provide.
Thanks
@Aditi_08
Could you please help me on how to get IP addresses of Site Crawler? Just please note, Site Crawler is working before November so IP addresses were not blocked.
Like it is mentioned before,
I am not sure why my reply not appearing here. Just for confirmation, replying again,
I like to confirm you -
There is no modification in Robots.txt
No issues with rate limit
Moz Pro settings are not changed
We are looking for your help to identify the issue.
Thanks
Thanks for your trouble shooting tips.
I assure you there has been nothing changed in robots.txt file or any settings in MozPro.
And there is frequency limit, Site Crawler triggers only once in 2 weeks.
Thanks
Hi,
In Moz pro, when using Site crawler (or recrawl), we are seeing message site is banned. But when using on-demand crawler, it could generate report successfully.
I just like to know if in both these cases, it is roberbot that is used!
And kindly note, site crawler was perfectly working before. So the required setup is already in place from long time. Site crawler ban issue started appearing from nov/dec 2023. .
Could you please us understand how could we possibly make site-crawler work?
I am happy to provide more details if you need any.
Thanks