SEO Web Crawler IP addresses
-
What are the IP addresses for the SEO Web Crawler?
There is a firewall on my clients website before it goes live, I would like to crawl the site before it goes live, but need to provide the web crawlers IP addreses.
Thank you for your time
-
Hi Jason,
This is Megan from the SEOmoz Help Team. Unfortunately, since we crawl from the cloud, RogerBot's IP address is changing all of the time so we cannot specify a particular IP address. Also, we do need the site to be live before we can crawl it, but you could remove the firewall and block all other bots except rogerbot so it won't get indexed anywhere else before it's ready.
I hope this helps! Please let me know if you have any other questions.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Help with best seo strategy
Hello all, I am a little bit confused about the "content suggestion'' feature in Moz. If I understood well, this feature suggests Anchor texts we can use in other page to link to the page we are trying to Optimize. So far, I've been using page keyword as Anchor Text in other articles. For example, I have a page about the best places to visit in Morocco, where the keyword is 'places to visit in Morocco'. I am using this same keyword as Anchor text in other pages to link to that page. While Moz suggests '10 best places' as Anchor text. What does it mean ? Is '10 best places' anchor text better ? Or should I use both ? Thanks in advance for your enlightenment.
Moz Pro | | sportygirl0 -
How do I combat title tag issues in Moz because of long company/site name? [Currently using Yoast SEO Plugin]
I've only just started to dip my toe into the SEO pool, so please forgive me if this is a silly question. I've been working heavily in the Moz tool and the Yoast SEO plugin to clean up the meta descriptions on our site, as there were many pages without descriptions in place (we just transferred from a different CMS to Wordpress within the last month or so). Now I'm noticing on our Site Crawl that we have 60+ pages that have title elements that are considered too long. When I go in to fix these issues on our blog posts (as this is where most of the issues lie), Yoast is already giving me the green light on the title lengths. I've discovered that the reason why Moz is telling us that many of our site lengths are too long is because "| Parker Staffing Services" is added to the end of our title tags automatically. Deep breath Now that you've heard all of the backstory... Is there a way that I can prevent that site title (which is 26 characters all by itself! Talk about limiting!) from displaying on search engines when our blog posts or pages show up in search results? I've poked around in the Titles & Metas settings that come with Yoast, but I fear making too many changes will result in me screwing something up. Help an SEO beginner out? 🙂
Moz Pro | | giaciccone0 -
Canonical URLs all show trailing slash on main site pages - using Yoast SEO for Wordpress - how to correct
We are using Yoast for a number of our sites. We use naked domain as the canonical. I have noticed in the header tags that all our sites show the canonical URLs as having a trailing slash: Example: http;//foxspizzajc.com, when I look at the source code, it shows the canonical as http;//foxspizzajc.com/ Of course, it is much more likely that all sites that link to us will not use the trailing slash - so preferably we do not want that to be the canonical - among other reasons. Does this need to be fixed so the trailing slash is removed? I cannot see how to do this in Yoast SEO or in Permalinks structure for Wordpress. Sorry for my ignorance. Thanks for any help.
Moz Pro | | Adam_RushHour_Marketing1 -
IP TRACKING
Is there a 3rd party that can provide me with all the ips that have hit my site http://1-800-medigap.com ? Does the MOZ API have that ability? Does anything?
Moz Pro | | jdcline0 -
SERPs ans SEO
I am trying to analyze my site to determine what we may be doing wrong as far as our site goes Ilike everyone else I suppose). In any event I have followed as much as I can from the Moz guideline and tools. We were ranking very well for quite some time and now..kaput. We have tried to make our site faster (using Google Page Speed ..the CDM for GOOGLE). We have also tried redoing out title and description tags to be more search and user friendly. Our site is very fast now...which is good. But we are still not seeing the rankings improvements that we had hoped for. We have built the site using WP and broke the template to make what we feel is an attractive site....used what we feel are the best plugins to capitalize....can anyone give us some feedback to let us know if we are just missing something here. We are RELEVANT to our industry...
Moz Pro | | AdsOnWheels0 -
Creating a SEO Report
We are looking to create a SEO report that is broken down by keywords. The traffic that the keywords generate for the site, the rankings in the search engines, the number of backlinks that have used the keyword as anchor text. We have a few tools that can do some of this, but are looking to find something that can aggregate all this info into a clean report. We are wondering if anyone knows a good website/application that can help manage a month-to-month report on the aspects above. Thanks!
Moz Pro | | insitegoogle0 -
SEOmoz crawler and duplicate content
Does anybody know if the SEOmoz crawler picks up canonical tags when its looking for duplicate content? I've got a ton of errors in one of my projects even though they all have canonical tags. Thanks!
Moz Pro | | neooptic0 -
Incorrect domain authority result on SEO tool bar and OSE
The SEO tool bar is returning what I believe to be an incorrect domain authority of 71 and showing 24,356,141 lins from 153,051 domains. The OSE is also returning 71 as domain authority. Anyone know what could be doing this? Thanks. Jason
Moz Pro | | jayderby0