SEO Web Crawler IP addresses
-
What are the IP addresses for the SEO Web Crawler?
There is a firewall on my clients website before it goes live, I would like to crawl the site before it goes live, but need to provide the web crawlers IP addreses.
Thank you for your time
-
Hi Jason,
This is Megan from the SEOmoz Help Team. Unfortunately, since we crawl from the cloud, RogerBot's IP address is changing all of the time so we cannot specify a particular IP address. Also, we do need the site to be live before we can crawl it, but you could remove the firewall and block all other bots except rogerbot so it won't get indexed anywhere else before it's ready.
I hope this helps! Please let me know if you have any other questions.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Unsolved ip address
hi, im currently running my web without domain, can i use moz with my web ip address. thanks
Moz Pro | | ikuraa0 -
Parked Domains and Local SEO
Hi, I wanted to know what your thoughts were on **Parked Domains **and Local SEO. I came across a company that has a lot of parked domains that were created for different countries. i.e. xxxxx.co.uk, xxxxx.ch, xxxxx.com etc. The content they load are from the main domain xxxxx.com I went to a country specific Google search, for instance google.co.uk and google.ch Searched for the company name and xxxxx.com showed up, not the parked domains that existed for each "country" No analytics exist as the tracking code is on xxxxx.com so I can't see if any specific traffic goes to xxxxx.ch, or other domains. From my understand, parked domains lead to unnecessary duplicate content. So why have them? Simply setup a redirect to xxxxx.com from the parked domain. But are there benefits for Local SEO? Search engines like google will still see the domain as a separate entity if no redirect is in place. If so, I realise content has to be unique..but it raises a question... Why have parked domains at all apart from stopping your competitors or users typing other variance of your URL? What do you guys think?
Moz Pro | | Diamond.Dog0 -
SEO and page redirects from a high ranking site quandary
We are launching a site on a new domain that is taking the place of a group (subset) of pages in an existing domain. BUT The pages on the existing domain have really good SEO rankings in a very competitive category and we want to leverage the traffic it gets today in the best way, so.... Which of the following would be the best practice in this case (in regards to SEO)? Modify the existing pages' content so that there are prominent calls to action that lead the users to the new domain. Create permanent redirects for the existing pages for their counterparts on the new domain. This is more direct for the user but we don't know how it will affect the current ranking. Something other than the above. Many thanks for you help Gary
Moz Pro | | gazza10 -
Web Site Migration Testing and SEO-QA Automation?
Hey Mozzers, Are there any good Migration-SEO-QA Tools out there? Given a prioritized list of URLs and prioritized list of Keywords, is there a tool that can compare basic SEO factors, old URL vs. new URL, and identify all the specific gaps that need to be fixed? Here is a basic SEO-QA acceptance checklist, for porting any website. . . . Until the porting work is completed we cannot accept the new website. Givens: 1. A list of the Top 100 URLs from the old site, prioritized by conversion rates, landing page traffic, and inbound links. 2. A list of the planned 404 - mapped URLs, old to new site, from the porting team. 3. A list of the current Top 200 Keywords, prioritized. 4. A good amount of SEO work has already been done, by several professionals, for the current (old) site. **How to evaluate if the new site will be acceptable to Google? Check ON-PAGE SEO Factors... ** **. . . that is, the NEW site must be AS GOOD AS (or better than) the current (old) site,
Moz Pro | | George.Fanucci
in the eyes of Google, to preserve the On-Page SEO work already done. ** Criteria: URLs ok? :: Is the URL mapping ok, old to new, best web page? LINKS ok :: Are all internal LINKS and keyword Anchor Text ported? TEXT ok :: On-page content, TEXT and keywords ok? TITLE ok :: HTML Title and title keywords ok? DESCRIPTION ok :: HTML Meta Description ok? H1, H2 ok :: HTML H1, H2 and keywords ok? IMG kwds :: HTML IMG and ALT keywords ok? URL kwds :: URL - keywords in new URLs ok? Potential porting defects: Keywords in URL missing: Keywords in HTML Title missing: Keywords in Meta Description missing: Any internal LINKS or Link anchor text missing: Keywords in Page TEXT missing: H1, H2 missing keywords: HTML IMG alt-text, IMG file URLs, any missing keywords: Notes: Until the porting work is completed we cannot accept the new site, or set a target date for potential cutover. There are eight (8) data items per URL, and about one hundred (100) URLs to be considered for SEO-QA before going live. We were expecting to cutover before the end of February, at the latest. There is no point in doing full QA acceptance-tests until the porting work is completed. QA spot-checks have found far too many defects. About 60% of the landing-page traffic comes via the top 40 URLs. With over 100 URLs to look at, it can take more than a week or two just to do SEO-QA in detail, manually, item-by-item, page-by-page, side-by-side, old vs. new. Spot-checks indicate a business disaster would occur unless the porting defects are fixed before going live. _Any Migration-QA Tools?_Given a prioritized list of URLs and prioritized list of Keywords, is there a tool that can compare basic On-Page SEO factors, old URL vs. new URL, and identify most of the specific gaps that need to be fixed before going live with the new site? _ *** Edit: Any comments on the SEO criteria, tools, or methods will be appreciated!_0 -
Can I see when SEO Moz has crawled my website?
I would like to know if it's possible to see (maybe in my Google Analytics) if SEO Moz has crawled my website. I'm also curious if and where I can see when the robot of Google visited my website. Thanks!
Moz Pro | | Spotler0 -
Automated SEO Report for Agencies
I have a lot of SEO reports to send out, SEO Moz is awesome but it's not designed for this kind of work. I would like an automated report that does an on page optimization report on the potential client's site, finding errors etc it does not have to include fixes for each. This is just a basic report but it must look nice and be user friendly. I have looked at Raven and it looks ok however it does not have any decent On Page Optimization reports. I know I could crawl and then delete the campaign in SEO Moz but that is very time consuming and far from ideal. If anyone knows of a solution please let me know 🙂
Moz Pro | | SEOKeith0 -
Top ranking with no SEO
I ran an SEOMoz keyword difficulty report for the term "interim CFO services" and found that all of the websites that ranked really well for that term have good page authority and domain authority scores and plenty of incoming links.....except 1. result #5 has a page and domain authority of 1 and no incoming links, however it's ranked #5 in google SERP for this highly competitive term. see screenshots: http://screencast.com/t/NJiOn8N4l http://screencast.com/t/x6tU1RhX4j Is this ranking based solely on the domain name being 100% relevant? Did they just get lucky and hit the google rankings jackpot? If this is the case, by the same logic I am trying achieve high rankings for the keyword "outsourced accounting firm" and purchased the domain name outsourcedaccountingfirm.com and am not even ranked in the top 50 with google for that keyword. (it's very new though) any feedback on either questions above would be greatly appreciated. Thanks!
Moz Pro | | texmeix0 -
Is there a recommended format when placing a business address on a webpage?
Hi All, I ask the question as I was trying to GeoTarget tool which happened to not recognise a business address I place in the footer on one of my sites. The tool states that including the address on page helps the search engines identify your location, so I'm curious whether a specific format works best when optimizing for local search? Thanks.
Moz Pro | | davebrown19750