How to get rogerbot whitelisted for application firewalls.
-
we have recently installed an application firewall that is blocking rogerbot from crawling our site. Our IT department has asked for an IP address or range of IP addresses to add to the acceptable crawlers. If rogerbot has a dynamic IP address how to we get him added to our whitelist? The product IT is using is from F5 called Application Security Manager.
-
Hi Joel,
Just wondering if you guys came up with a fix for this one? I have the same issue myself...
Thanks!
-
We have a similar issue where we block the Amazon cloudspace but want to allow rogerbot. Our firewall finds it much easier to block by IP range rather than user-agent (rogerbot).
Can you please tell us more about the solution you suggested?
-
Hi, our IP is changing all the time. We do have an option that may resolve your issue though. I'm going to create a ticket as I will need some personal information regarding your account.
Thanks,
Joel
-
Hi kgoss -
I'm not an SEOmoz employee - but just fyi, here are the IP addresses rogerbot used when accessing my site since the start of 2012:
50.17.63.219
184.72.211.58
50.19.77.29
174.129.177.94
184.73.93.243
107.20.64.186
23.22.26.208
107.22.19.89
50.19.188.211
50.19.195.34
107.22.107.114
107.22.137.134
216.244.72.3
75.101.205.127
67.202.42.32
216.176.191.232
23.20.247.207
216.244.72.9
23.22.48.203
216.176.191.234
184.73.13.203
107.20.11.69
50.17.96.67
184.73.137.220
184.73.22.42
184.72.175.148
67.202.13.192
23.21.33.19
107.22.0.173
107.22.157.76
50.16.164.143
50.16.43.91
107.21.143.222
216.176.191.201
107.20.91.74
50.16.88.111
204.236.213.120
184.73.118.141
50.17.38.180
107.21.150.218
216.244.72.10
216.244.72.11
216.244.72.12
23.22.53.56Hope that helps.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Can anyone offer an example of a site or page that gets 100% or even close to that on Search Visibility?
I have a couple of sites that I manage that kill it in the SERPs and yet they get a low search visibility score from Moz Pro, I am talking 18%-19%, and another that ranks well has a search visibility score of 8.71%. I know there are factors that go into calculating the score, I am just curious if anyone is really up there.
Moz Pro | | -b.graves- 00 -
Do you get link equity from a redirect service?
When using a redirect service, do you get link equity from the redirect service or only from the linking website? Using opensite explorer in Moz, always shows me directory and redirect services as most important contribution for the domain authority. But I'm questioning if this is true. P.S. I'm not asking if link equity is passed from the linking website. That question has been answered many many many times. (hint: the answer is yes). H58xHAl.png pGH7e48.png
Moz Pro | | jasny0 -
My homepage is not getting indexed by google for some reason
My homepage http://www.truebluelifeinsurance.com is not indexed by google. The rest of my site is indexed. The hompage is indexed by bing. I looked in the webmaster tools and there is no indication why. I believe the issue started when I did a site re-design in August. Any ideas?
Moz Pro | | Brian_Greenberg0 -
Allow only Rogerbot, not googlebot nor undesired access
I'm in the middle of site development and wanted to start crawling my site with Rogerbot, but avoid googlebot or similar to crawl it. Actually mi site is protected with login (basic Joomla offline site, user and password required) so I thought that a good solution would be to remove that limitation and use .htaccess to protect with password for all users, except Rogerbot. Reading here and there, it seems that practice is not very recommended as it could lead to security holes - any other user could see allowed agents and emulate them. Ok, maybe it's necessary to be a hacker/cracker to get that info - or experienced developer - but was not able to get a clear information how to proceed in a secure way. The other solution was to continue using Joomla's access limitation for all, again, except Rogerbot. Still not sure how possible would that be. Mostly, my question is, how do you work on your site before wanting to be indexed from Google or similar, independently if you use or not some CMS? Is there some other way to perform it?
Moz Pro | | MilosMilcom
I would love to have my site ready and crawled before launching it and avoid fixing issues afterwards... Thanks in advance.0 -
Adding new categories to my website and getting them ranked
Hi, I currently run a website and have added a couple of new categories lately. The website is Planes Trains Automobiles Co UK. For example I want to add "Planes for sale" as a new category. What is the best way to go about this? Will each aircraft added add content therefore ranking? I don't want to approach an SEO company yet as the service would be free for the time being? Thanks
Moz Pro | | PlanesTrainsAutos0 -
My page has a 302 redirect and I don't know how to get rid of it!
I use Internet Officer tool to see the 302 redirect but I check the redirects in the CPanel and there are none. In the .htaccess there are none either. I don't know where else to look 😞 The url is http://servicioshosting.com Can you guys help me? I can't set up a campaign because Google can't crawl the website. I can't setup the Facebook OpenGraph because of the redirect. error.jpg
Moz Pro | | vanessacolina0 -
Getting SEOMoz reports to ignore certain parameters
I want the SEOMoz reports to ignore duplicate content caused by link-specific parameters being added to URLs (same page reachable from different pages, having marker parameters regarding source page added to the URLs). I can get Google and Bing webmaster tools to ignore parameters I specify. I need to get SEOMoz tools to do it also!
Moz Pro | | SEO-Enlighten0 -
How often does SEOmoz reports get refreshed?
Is there a way I can refresh the reports manually instead of waiting for it to pull the updated data?
Moz Pro | | RBA0