Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
SEO Web Crawler IP addresses
-
What are the IP addresses for the SEO Web Crawler?
There is a firewall on my clients website before it goes live, I would like to crawl the site before it goes live, but need to provide the web crawlers IP addreses.
Thank you for your time
-
Hi Jason,
This is Megan from the SEOmoz Help Team. Unfortunately, since we crawl from the cloud, RogerBot's IP address is changing all of the time so we cannot specify a particular IP address. Also, we do need the site to be live before we can crawl it, but you could remove the firewall and block all other bots except rogerbot so it won't get indexed anywhere else before it's ready.
I hope this helps! Please let me know if you have any other questions.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved ip address
hi, im currently running my web without domain, can i use moz with my web ip address. thanks
Moz Pro | | ikuraa0 -
SEO impact of redirecting high ranking mirror site to the main website
During SEO audit for a client I noticed that they had over a dozen duplicate websites that are carbon copies of the main website. This was done via CMS platform and DNS. One of the mirror sites has about 400 indexed pages and has Moz DA of 42 and 137k External Equity-Passing Links. Full metrics comparison is attached. I originally planned on doing rel="canonical" on the mirror site but the CMS vendor never even heard of it and is refusing to implement it in the header. My only other option is doing one to one 301 redirects. Since the mirror site ranks well, even competes with main domain for some positions on the 1st page of SERP, what will be the impact after the redirects? Is doing 301's still the best option? Thanks! PrUpN3q
Moz Pro | | dasickle0 -
Will Moving Categories Affect SEO
I recently designed a new website to replace an old site. We managed to hold 90% ranking and traffic by keeping the same URS's and content. Now that we have completed that we are now updating the whole site. It is an commerce website. Some of the items we were selling we are getting from a new vendor. If I move these products from one one vendor to another will this affect SEO? Here is an example. I have a product called "Green Zipper Sweater". This product is Anchored via Manufacturer and Category. The URL and Title Tag are green-zipper-sweater. If the Sweater was made by "Nike Green Sweaters" and our new supplier is "Gap Yellow and Green Sweaters". If I change the the Manufacturer and now put it under "Gap Green and Yellow Sweaters" will this affect my ranking. We are continuing to stock products from the original supplier "Nike Green Sweaters" and we have an aggressive SEO plan and are ranking very well for "Nike Green Sweaters". We also have good product ranking so "Green Zipper Sweater" brings us a lot of traffic. I want to be sure I do not loose ranking for the product page "Green Zipper Sweater" and the Brand "Nike Green Sweaters" Any advice would be appreciated.
Moz Pro | | robbieire0 -
Mac Alternatives for Netpeak & SEO Tools For Excel?
Does anybody know of any mac alternatives for Netpeak & SEO tools for excel? I haven't been able to find any. I just need something to pull PA & DA quickly for a list of domains and URLs. Will I just have to create something custom with the Moz API?
Moz Pro | | kking41200 -
Crawlers crawl weird long urls
I did a crawl start for the first time and i get many errors, but the weird fact is that the crawler tracks duplicate long, not existing urls. For example (to be clear): there is a page: www.website.com/dogs/dog.html but then it is continuing crawling:
Moz Pro | | r.nijkamp
www.website.com/dogs/dog.html
www.website.com/dogs/dogs/dog.html
www.website.com/dogs/dogs/dogs/dog.html
www.website.com/dogs/dogs/dogs/dogs/dog.html
www.website.com/dogs/dogs/dogs/dogs/dogs/dog.html what can I do about this? Screaming Frog gave me the same issue, so I know it's something with my website0 -
Does anyone have an SEO strategic plan template for a beginning SEOmoz'r?
I'm fairly new to the SEO world and am going to be creating an SEO strategic plan for my boss (with his help). I was wondering if anyone had any recommendations on a template to use, outline to follow, must have's/don't need, etc. Anything you're willing to share would be greatly appreciated!
Moz Pro | | kyle21ct0 -
What user agent is used by SEOMOZ crawler?
We have a pretty tight robots.txt file in place to only allow the major search engines. I do not want to block SEOMOZ.ORG from being able to crawl the site so I want to make sure the user agent is open.
Moz Pro | | eseider0 -
Is having the company's address in the footer (or header) of each webpage important for SEO?
Is having the company/office's address in the footer (or header) of each webpage important for SEO? Your page for the Geotarget tool says that having the address in this element helps search engines find your location. My question is, how important or relevant is this to SEO? How does knowing the address influence SEO? Is it best SEO practice to put the address in the footer of every webpage? http://www.seomoz.org/geotarget
Moz Pro | | richardstrange0