SEO Web Crawler IP addresses
-
What are the IP addresses for the SEO Web Crawler?
There is a firewall on my clients website before it goes live, I would like to crawl the site before it goes live, but need to provide the web crawlers IP addreses.
Thank you for your time
-
Hi Jason,
This is Megan from the SEOmoz Help Team. Unfortunately, since we crawl from the cloud, RogerBot's IP address is changing all of the time so we cannot specify a particular IP address. Also, we do need the site to be live before we can crawl it, but you could remove the firewall and block all other bots except rogerbot so it won't get indexed anywhere else before it's ready.
I hope this helps! Please let me know if you have any other questions.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved Why does Moz Crawler start with HTTP//1.1 version??
We have run Moz Pro's Site Crawl for example-domain.com Why has Moz's crawler decided to site http://www.example-domain.com (ie the non-secure version) as zero crawl depth page and report the actually live https://www.example-domain.com (ie secure HTTP2 version) as a crawl depth of 2? Surely the main live page should be the first crawled and reported as crawl depth = 0?
Moz Pro | | AKCAC0 -
Looking for an SEO & MOZ Pro to add to my team!
Hi community, I'm a long-time MOZer and SEO director for a San Francisco-based digital marketing company providing complete end-to-end online growth solutions...and I'M HIRING! If you've got at least 3 years SEO experience and know MOZ inside and out, please reach out to me.
Moz Pro | | upgrowsf0 -
What SEO tools do you use in conjunction with Moz?
It seems like most people use multiple SEO tools. I am interested in hearing what you use in conjunction with Moz and why. -Stephen
Moz Pro | | martechwiz2 -
WIX - problem with On-Page Optimization picking up SEO
Hi, I'm new to here so please be gentle!
Moz Pro | | sportinglists
I have a WIX site and had it for a few months. I have m own domain sportinglists.com and I am using MOZ to improve the SEO. Having read a few things it seems that I may struggle, given that I have a WIX site. I am playing around with a certain page, that will most likely be the most popular one on the site (fingers crossed), however when I run On-Page Optimization it fails to pick up the following things that I know I have added to the page; H1 (page title) - I added through the text options
Body text - I added paragraph text and
Image Alt - put in through the settings Is this because that it is made in WIX that this won't be picked up or is more likely my incompetency? Thanks0 -
Is there an SEO checklist that I can follow for every new project?
I'm a very structured dude. I literally have to have every project in front of me in order to be productive. With that said, it would sure help me if there was a checklist to follow with at least the basics to both audit an existing website, to where they rank and what areas need improved and then another checklist that goes through each phase of implementing SEO for the first time either on a brand new website or an existing website that you're taking over... Any suggestions?
Moz Pro | | 2Spurs0 -
Can I base a SEO campaign on a specific MLS listing inteads of full broker web site?
I have been ask to do a SEO campaign for one MLS listing: house for sale in the rosedale area in Toronto, and I am having difficulty to target one specific MLS listing . I don't want to do the SEO campaign on http://jimmymolloy.com but rather one specific listing http://jimmymolloy.com/feature/?mls=A_C2337646&listing=for-sale Google Analytic does not allow me to have a URL with a query.
Moz Pro | | pd20120 -
Possible to have more then one SEO Moz login / account
We have an SEO Moz subscription and we were wondering if there was a way to setup multiple users attached to that subscription (i.e. separate logins).
Moz Pro | | Panjiva2 -
How can I view the "Ultimate Advanced SEO Bundle" online after purchasing it?
I purchased the Ultimate Advanced SEO Bundle DVD set and I understand that I can also watch it online. Where do I have to go to view it? MC
Moz Pro | | deuce1s0