What is the IP address of the MOZ crawlbot?
-
Our preproduction site is restricted to a specific IP range.
Prelaunch I'd like to use MOZ to check the site but this obviously isn't currently possible. Is there an IP range I can whitelist ?
-
Hi,
Moz is hosted on Amazon. See this sample log entry from today:
54.146.235.41 - - [17/Sep/2017:09:48:13 +0200] "GET /best-deals.html/page/12/ HTTP/1.0" 200 15803 "-" "rogerbot/1.2 (https://moz.com/help/guides/moz-procedures/what-is-rogerbot, rogerbot-crawler+aardwolf-production-crawler-97@moz.com)"[XF 54.146.235.41]
That IP is part of Amazon's ASN.
whois -h whois.radb.net -- '-i origin AS16509' | grep ^route
You can either use something like mod_authz to allow the domain amazonaws.com to connect or you can get each IP from AS16509 and allow that.
Hope it helps.
Regards
Chris
-
Just to expand on this, I found the following quote from a Moz member of staff:
"I wish I could give you IP addresses, but they change all the time since we host Roger in the cloud. There's not even a reliable range of IPs to give you. You can totally whitelist the useragent rogerbot, but that's the only reliable information about the crawler you can go off of. I hope that helps but let me know if there's any other solution you can think of. Thank you! "
From a previous question: https://moz.com/community/q/is-there-a-whitelist-of-the-rogerbot-ip-addresses
-
Unfortunately this is all I can find: https://moz.com/help/guides/moz-procedures/what-is-rogerbot with no IP range shown.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
I used moz tool and it was very useful to me.
I used moz tool for 3 months. Before that my website never ranked. But now it started ranking. Thanks to moz. My website is https://www.designcafe.com
Getting Started | | designcafe1 -
Moz Not Crawling Angular SPA
I have a client that just launched a redesigned website using Angular as a single page app. Google appears to be able to crawl the site just fine, but Moz crawl is only finding one page. We have updated the htaccess to allow for Rogerbot and Dotbot, but still unable to crawl any pages other than the home page. Does anyone have experience with this or ideas of why it won't crawl all pages, and how to allow for Moz to crawl all pages? There is a sitemap with approx. 390 pages. Thanks!
Getting Started | | PIN_Celler1 -
How to have MOZ site crawl pre-launch
Hi, Our new website is about to launch. We would love to have moz.com SITE CRAWL our site before launch. For issues like "missing meta description" and everything else that moz.com checks. We would love to do it before we launch. The new site is currently on a different domain than our live site. example.com <-- this is our live site. new-site.com <-- this is our "staging" server with the new site. We have a long running campaign for example.com Do we need to create a new campain for new-site.com ? Or is there some other simpler way? When we launch we will switch the site from new-site.com to example.com .. example.com will be the address for the new site.. Any ideas or suggestions? best practices? edit Forgot to say thank you for your help and input 🙂
Getting Started | | tandvarden0 -
Anyone use On-Page Grades in Moz? Please Give us Feedback!
Jackie from Moz here. We're looking to get some feedback on the letter grades we use for On-Page Grades (in the application and in the Research Tools). We made this super easy survey to collect our Community's thoughts. Tell us what you think! Thanks for all your help!
Getting Started | | JackieRae2 -
Why Moz New Users Are Left Alone?
New users when sign up to an application like Moz need concrete support to make them familiarized the basics at least, but Moz seems to be not realizing this hardship of the new users or intentionally ignoring it. There is no chat service or phone numbers to quickly contact the designated agent for some quick help. Mail responses that are received after much time have their limitations -- not much interactive. What the new users should do, because the Moz is not that simple for new users. [edited question for formatting]
Getting Started | | Sequelmed1 -
MOZ for initial keyword research?
Can I use MOZ for initial keyword research? Can't create an AdWords campaign (no website yet to register) to get use of Google's Keyword Planner & my subscription to Wordtracker is showing me too many unrelated keywords... Peter
Getting Started | | nodiffrei0 -
Why can't I add more than one campaign in my trial version of MOZ?
Hi, I'm on the 30 day trial and I can only run one campaign right now. I thought I could add in 5 campaigns. When I go into Manage Campaigns the 'add a campaign' button is light blue and I can't click it. Is this just because I'm on the trial? Or should I be able to add 4 more campaigns?
Getting Started | | Sophie-Kool0 -
Lost since the change over to Moz
I was recently lead to a series of videos which looked to be quite helpful here http://moz.com/help/pro/introduction But is seems with the change over from SEO Moz to Moz, nothing is in the same place. Should I bother watching them or wait for new ones? I'm particularly interested in SEO to raise our rank.
Getting Started | | AliciaMarie0