What is the IP address of the MOZ crawlbot?

[mrphome]

Our preproduction site is restricted to a specific IP range.

Prelaunch I'd like to use MOZ to check the site but this obviously isn't currently possible. Is there an IP range I can whitelist ?

[chrislemmer]

Hi,

Moz is hosted on Amazon. See this sample log entry from today:

54.146.235.41 - - [17/Sep/2017:09:48:13 +0200] "GET /best-deals.html/page/12/ HTTP/1.0" 200 15803 "-" "rogerbot/1.2 (https://moz.com/help/guides/moz-procedures/what-is-rogerbot, rogerbot-crawler+aardwolf-production-crawler-97@moz.com)"[XF 54.146.235.41]

That IP is part of Amazon's ASN.

whois -h whois.radb.net -- '-i origin AS16509' | grep ^route

You can either use something like mod_authz to allow the domain amazonaws.com to connect or you can get each IP from AS16509 and allow that.

Hope it helps.

Regards

Chris

[ThomasHarvey]

Just to expand on this, I found the following quote from a Moz member of staff:

"I wish I could give you IP addresses, but they change all the time since we host Roger in the cloud. There's not even a reliable range of IPs to give you. You can totally whitelist the useragent rogerbot, but that's the only reliable information about the crawler you can go off of. I hope that helps but let me know if there's any other solution you can think of. Thank you! "

From a previous question: https://moz.com/community/q/is-there-a-whitelist-of-the-rogerbot-ip-addresses

[ThomasHarvey]

Unfortunately this is all I can find: https://moz.com/help/guides/moz-procedures/what-is-rogerbot with no IP range shown.