Welcome to the Q&A Forum

Browse the forum for helpful insights and fresh discussions about all things SEO.

Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you have been placed in read-only mode.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

Best way to block spambots in htaccess

Reporting & Analytics

962

ijb last edited by

I would like to block Russian Federation, China and Ukraine spam as well as semalt and buttonsforwebsite. I have come up with the following code, what do you think?

For the countries:

BLOCK COUNTRY DOMAINS

RewriteCond %{HTTP_REFERER} .(ru|cn|ua)(/|$) [NC]
RewriteRule .* - [F]

And for buttons-for-website.com and semalt-semalt.com:

BLOCK REFERERS

RewriteCond %{HTTP_REFERER} (semalt|buttons) [NC]
RewriteRule .* - [F]

or should it be:

BLOCK USER AGENTS

RewriteCond %{HTTP_USER_AGENT} (semalt|buttons) [NC]
RewriteRule .* - [F]

Could I add (semalt|buttons|o-o-6-o-o|bestwebsitesawards|humanorightswatch) or is that too many?
1 Reply Last reply
Reply Quote 0
donford last edited by

Hi

I think you're on the right track.

A very good blog post by Jared Gardner has recently addressed this question on the Moz Blog.

Hope it helps,

Don
1 Reply Last reply
Reply Quote 1

Got a burning SEO question?

Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.

Start my free trial

Browse Questions

View

From

Sorted by

With category

Explore more categories

Related Questions

Is there a way to filter all computers on a specific IPv6 network in Google Analytics?

Is there a quick way of filtering the IP addresses for all the computers on a network that's using IPv6? I want to filter out visits to our websites from the devices on our office network, but each computer (and phone and tablet) seems to have a different address. It _looks _like they all start the same way, though. One computer is xxxx:xxxx:xxxx:xxxx:aaaa:aaaa:aaaa:aaaa, another is xxxx:xxxx:xxxx:xxxx:bbbb:bbbb:bbbb:bbbb, my phone is xxxx:xxxx:xxxx:xxxx:cccc:cccc:cccc:cccc, etc. Does this mean that xxxx:xxxx:xxxx:xxxx is the address for our network as a whole, and I can just set up a Google Analytics filter for "IP addresses starting with..."? Or would doing that also filter out hits from, like, every visitor within a 20 mile radius of our office? If I need to simply put in the individual addresses for each and every device, I will. I'm just hoping it doesn't come to that. Thanks!
Reporting & Analytics | | BrianAlpert78

0
Best way to handle duplicate title on Home page?

Moz reports two links to the same Home page ad duplicate titles ... http://myhjhome.com/index.php
http://myhjhome.com I'm not sure if I should just 301 redirect http://myhjhome.com/index.php to http://myhjhome.com, or if there is a better way to handle that? Or should I comb the website and make sure all links to the Home page dont include index.php? Just looking for some extra help here, learning as I'm going, thanks!!
Reporting & Analytics | | ElykInnovation

0
800,000 pages blocked by robots...

We made some mods to our robots.txt file. Added in many php and html pages that should not have been indexed. Well, not sure what happened or if there was some type of dynamic conflict with our CMS and one of these pages, but in a few weeks we checked webmaster tools and to our great surprise and dismay, the number of blocked pages we had by robots.txt was up to about 800,000 pages out of the 900,000 or so we have indexed. 1. So, first question is, has anyone experienced this before? I removed the files from robots.txt and the number of blocked files has still been climbing. Changed the robots.txt file on the 27th. It is the 29th and the new robots.txt file has been downloaded, but the blocked pages count has been rising in spite of it. 2. I understand that even if a page is blocked by robots.txt, it still shows up in the index, but does anyone know how the blocked page affects the ranking? i.e. while it might still show up even though it has been blocked will google show it at a lower rank because it was blocked by robots.txt? Our current robots.txt just says: User-agent: *
Disallow: Sitemap: oursitemap Any thoughts? Thanks! Craig
Reporting & Analytics | | TheCraig

0
What is best practice for tracking RSS feed subscribers

What is the most accurate/achievable way of tracking data about subscribers to your RSS feed through Google Analytics? With standard WordPress sites, we place the RSS link to Feedburner so we could track statistics. However it wouldn't track the way that I use it. I use Pulse on an Android Tablet to read my feeds offline on the bus each morning. At home, Pulse automatically downloads the latest feeds wirelessly overnight. So then I can read them without a connection. The obvious downside for my reading experience is that I only get what is contained in the feeds. If the company only includes an excerpt, it's too annoying to read the teaser and be unable to connect and follow a link. So I only subscribe to feeds that contain the full post. Yeah to seomoz, aimclear, SEL, adwordsblog. I dont subscribe to bruceclays blog, much as i'd like to, because it doesn't contain the full feed. That's probably deliberate on their part, because I have to consciously visit their blog on my desktop at work, to see the whole post. The other problem with say Pulse, is how it locates the feed. I typed in the URL, and Pulse subscribed me. I assume that Pulse simply looked for the domain.com/feed URL and added that, rather than look for feeds2.feedburner.com/domain. I looked at Feedburner stats and they didn't go up for 2 days, so basically it didn't track me. Would it be as simple as using the Google URL builder to add parameters to each post in the RSS feed? Eg utm_source=feedreader, utm_medium=rss, utm_campaign=tracking. But that still wouldn't track offline users. I assume that most people are also not going to paste the Feedburner URL into their FeedReader, but would let the platform auto-detect the feed. Any suggestions?
Reporting & Analytics | | ozgeekmum

1
Is there any way to see how one my keywords ranked historically before I started tracking it?

Hello there! I'm in need of some historical data on some keywords as they relate to my site. Basically, I'd like to track which events have had the most impact on moving me through the rankings, but I wasn't using Moz at the time I had made some changes to my website... Is there any way to see how I ranked for a particular keyword at a given point in time? Thanks! Gene
Reporting & Analytics | | BGroup

0
Best practice SEO/SEM/Analaytics/Social reports

Hi All, does anyone have a best practice excel spreadsheet of a internal report we should be using.... ie what are the main factors we should be tracking? Unqiue views? time spent on site? Where they came from? seo/sem/network/direct to site? social media tracking? amount of +1/fb likes/tweets etc thanks
Reporting & Analytics | | Tradingpost

0
What is the best Wordpress Analytics Plugin to use?

I am installing a new instance of WordPress and want to use Google Analytics. Which Plug-in is the best to use? I am finding several Plug-ins that all seem to do the same thing: Google Analyticator Google Analytics Dashboard Simple Google Analytics Ultimate Google Analytics Wordpress Google Analytics Maybe one of these are not the best? Any recommendations would be appreciated. Thanks!
Reporting & Analytics | | LBike

0
What analytics program is best for a small business?

Of course Google analytics is good. I tried getclicky.com. Are there any other analytics programs that are better that are also affordable? What do you use?
Reporting & Analytics | | DallasBonsai

0