SEO Web Crawler IP addresses
-
What are the IP addresses for the SEO Web Crawler?
There is a firewall on my clients website before it goes live, I would like to crawl the site before it goes live, but need to provide the web crawlers IP addreses.
Thank you for your time
-
Hi Jason,
This is Megan from the SEOmoz Help Team. Unfortunately, since we crawl from the cloud, RogerBot's IP address is changing all of the time so we cannot specify a particular IP address. Also, we do need the site to be live before we can crawl it, but you could remove the firewall and block all other bots except rogerbot so it won't get indexed anywhere else before it's ready.
I hope this helps! Please let me know if you have any other questions.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved Why does Moz Crawler start with HTTP//1.1 version??
We have run Moz Pro's Site Crawl for example-domain.com Why has Moz's crawler decided to site http://www.example-domain.com (ie the non-secure version) as zero crawl depth page and report the actually live https://www.example-domain.com (ie secure HTTP2 version) as a crawl depth of 2? Surely the main live page should be the first crawled and reported as crawl depth = 0?
Moz Pro | | AKCAC0 -
Mobile SEO Rank Tracker - Hourly
Hello folks! Does anyone have a suggestion for a rank tracker that can track high-profile keywords, in mobile specifically, on an hourly basis? Thanks!
Moz Pro | | rvseo0 -
Who else has inaccurate reporting? SEO Moz BUG
I have been in contact with support to little avail no timelines for the correction kind of leaves me out in the cold. I have loved SEOmoz in the past but this recent bug is costing me time and money. Can someone please tell me how they are working around all these issues in rank reporting and a zero accuracy rate on page report cards? bugs are holding up my SEO efforts. SEOmoz has been in touch but has provided no timelines. My programs are suffering. I found inaccuracy in each report metric and now have very little faith that this is going to be corrected in a timely fashion. I rely greatly on the wonderful tools SEOmoz provides and now I am flying blind. Who else has been hit by the bug? And, what has support done for you to help?
Moz Pro | | ericajane0 -
SEO process suggestions / keyword analysis
hi all, we've done a bit of SEO over the years for our site, and are looking to pick things up this year and spend more time on SEO. first off, we are working on revising our keyword list & would like to view what keywords specific competitors are ranking for and try to target those keywords as well. is this possible via SEOmoz (similar to spyfu)? please let me know if you feel this is a good first step in picking up our SEO efforts for the new year. i've been looking at the research tools & is seems the Keyword Difficulty Tool hasn't been fully functional for the past week or so ("Our Keyword Difficulty Tool is having issues displaying AdWords API data. We are working with the AdWords API folks to get a fix soon!"). also, i noticed i can enter competitor sites in open site explorer & compare link metrics, but i don't see anywhere to compare keyword metrics. is there something i'm missing? thanks again everyone!
Moz Pro | | lsat0 -
Onsite SEO tools
i have been working on an SEO site and I wanted to add a website checker tool to my home page to help keep users on my site for longer and start interacting. Are there any available scripts for a website tester that I can place onto my home page? This way when a user lands on my website they can check their "score" and then they can read more about how we came up with this number or what they can do to help it etc..
Moz Pro | | SEODinosaur0 -
How can I create multiple private labeled website SEO analysis reports?
How can I create multiple white-labeled website SEO analysis reports using the SEOMoz software? I'm speaking at a workshop and want to provide this as a giveaway to attendees to start the discussion on improving their SEO. I'm looking mostly for an overview summary of a site's key SEO data points including inbound links, on-page SEO details including page titles, and basic recommendations for improving. Websitegrader.com offers this but not white-labeled. I'm not sure about how SEOMoz handles this. Thanks
Moz Pro | | bshanahan0 -
SEO PowerSuite - best kept SEO secret or just bullsh..?!?
I've come across a lot of SEO tools in the last 5-6 years. And there are many great tools out there! I don't believe in the one-tool-fits-all. Among the tools I make use of are Raven Tools and Searchmetrics (great audit and market analysis tool!), and the tools in here, of course. In my search for a better ranking report tool I came across Rank Tracker from SEO PowerSuite. That seems like a great tool in category with AWR; but I am not able tell so as I am not a power user. Well, here is where I am going: I then tested the rest of the PowerSuite and it seems as a great product. But searching for the tools in here, and nothing shows up. Silence. Also elsewhere in other serious places. Why is that? It is a collective secret, shhhh, don't tell 🙂. Or is it is just crap? I know they have a bad reputation in regard to prices and updates. What do you guys think?
Moz Pro | | Danni0 -
SEOMoz site crawlers created an issue for our servers
I have set up a number of campaigns with your pro tool. Unfortunately we have 7 sites on our server and our IT dept have said that we had an issue when your site crawlers visited for several sites at the same time - is there any way that I can retain the campaigns but have the sites crawled on request rather than automatically?
Moz Pro | | StephenALee0