Best way to block spambots in htaccess
-
I would like to block Russian Federation, China and Ukraine spam as well as semalt and buttonsforwebsite. I have come up with the following code, what do you think?
For the countries:
BLOCK COUNTRY DOMAINS
RewriteCond %{HTTP_REFERER} .(ru|cn|ua)(/|$) [NC]
RewriteRule .* - [F]And for buttons-for-website.com and semalt-semalt.com:
BLOCK REFERERS
RewriteCond %{HTTP_REFERER} (semalt|buttons) [NC]
RewriteRule .* - [F]or should it be:
BLOCK USER AGENTS
RewriteCond %{HTTP_USER_AGENT} (semalt|buttons) [NC]
RewriteRule .* - [F]Could I add (semalt|buttons|o-o-6-o-o|bestwebsitesawards|humanorightswatch) or is that too many?
-
Hi
I think you're on the right track.
A very good blog post by Jared Gardner has recently addressed this question on the Moz Blog.
Hope it helps,
Don
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Help Blocking Crawlers. Huge Spike in "Direct Visits" with 96% Bounce Rate & Low Pages/Visit.
Hello, I'm hoping one of you search geniuses can help me. We have a successful client who started seeing a HUGE spike in direct visits as reported by Google Analytics. This traffic now represents approximately 70% of all website traffic. These "direct visits" have a bounce rate of 96%+ and only 1-2 pages/visit. This is skewing our analytics in a big way and rendering them pretty much useless. I suspect this is some sort of crawler activity but we have no access to the server log files to verify this or identify the culprit. The client's site is on a GoDaddy Managed WordPress hosting account. The way I see it, there are a couple of possibilities.
Reporting & Analytics | | EricFish
1.) Our client's competitors are scraping the site on a regular basis to stay on top of site modifications, keyword emphasis, etc. It seems like whenever we make meaningful changes to the site, one of their competitors does a knock-off a few days later. Hmmm. 2.) Our client's competitors have this crawler hitting the site thousands of times a day to raise bounce rates and decrease the average time on site, which could like have an negative impact on SEO. Correct me if I'm wrong but I don't believe Google is going to reward sites with 90% bounce rates, 1-2 pages/visit and an 18 second average time on site. The bottom line is that we need to identify these bogus "direct visits" and find a way to block them. I've seen several WordPress plugins that claim to help with this but I certainly don't want to block valid crawlers, especially Google, from accessing the site. If someone out there could please weigh in on this and help us resolve the issue, I'd really appreciate it. Heck, I'll even name my third-born after you. Thanks for your help. Eric0 -
Is there a way to apply the same google analytics filter to multiple properties?
I manage WordPress multiple sites for my clients. Some of these clients are SEO customers. One issue I have with the analytics reports is the occurrence of spam and ghost spam. I know how to create filters to block however there are always more and more. Is there away to export the filter and import it to all the other properties i manage? Or do they just have to be done manually every time i need to block spam?
Reporting & Analytics | | donsilvernail1 -
What is the best way to embed Google Analytics charts on our site?
We want to build out this functionality so a client can log into our site and view data. I know its possible but I can't find any articles about the steps so that my team can move forward.
Reporting & Analytics | | appbackr0 -
Best top 5 actionable insights from Google Analytics
Buongiorno from 14 degrees C cloudy wetherby UK 🙂 With a whole gamut of books out there are written around the topic of "Actionable Web analytics" where you have to wade through chapters of how to suck eggs and how to get analytics part of your business culture (whatever! most clients who look after their websites are entry level Marketing Execs who pay the digital agencies to do the thinking for them) I'd like to cut to the chase and list out say a top 5 of actionable insights a client would actually give the time of day for so heres my top five, whats yours? 1. Landing page bounce - showing them the £5k landing page designed with no conversion thinking is turning off customers
Reporting & Analytics | | Nightwing
2. Synching up PPC campaigns to analytics and showing them how much money they are waisting
3. Goal funnels - Showing them there 20 step shopping conversion funnel gets abandoned at step 3
4. Non brand keyword traffic (although a stack of this gets blocked by "not provided" Big groan...)
5. Event tracking on there 10 page carousel anner showing them no one clicks on banners 2 - 10 So under the strict label of "Actionable data only" from Google analytics whats your best insights you share with clients in the hope that they may actually give data reports the time of day! Grazie tanto,
David1 -
How many users completely block Google Analytics cookies ?
Hello everyone! In your experience, how many of your visitors' browsers completely block cookies including those of Google Analytics ?
Reporting & Analytics | | Masoko-T0 -
800,000 pages blocked by robots...
We made some mods to our robots.txt file. Added in many php and html pages that should not have been indexed. Well, not sure what happened or if there was some type of dynamic conflict with our CMS and one of these pages, but in a few weeks we checked webmaster tools and to our great surprise and dismay, the number of blocked pages we had by robots.txt was up to about 800,000 pages out of the 900,000 or so we have indexed. 1. So, first question is, has anyone experienced this before? I removed the files from robots.txt and the number of blocked files has still been climbing. Changed the robots.txt file on the 27th. It is the 29th and the new robots.txt file has been downloaded, but the blocked pages count has been rising in spite of it. 2. I understand that even if a page is blocked by robots.txt, it still shows up in the index, but does anyone know how the blocked page affects the ranking? i.e. while it might still show up even though it has been blocked will google show it at a lower rank because it was blocked by robots.txt? Our current robots.txt just says: User-agent: *
Reporting & Analytics | | TheCraig
Disallow: Sitemap: oursitemap Any thoughts? Thanks! Craig0 -
What is best practice for tracking RSS feed subscribers
What is the most accurate/achievable way of tracking data about subscribers to your RSS feed through Google Analytics? With standard WordPress sites, we place the RSS link to Feedburner so we could track statistics. However it wouldn't track the way that I use it. I use Pulse on an Android Tablet to read my feeds offline on the bus each morning. At home, Pulse automatically downloads the latest feeds wirelessly overnight. So then I can read them without a connection. The obvious downside for my reading experience is that I only get what is contained in the feeds. If the company only includes an excerpt, it's too annoying to read the teaser and be unable to connect and follow a link. So I only subscribe to feeds that contain the full post. Yeah to seomoz, aimclear, SEL, adwordsblog. I dont subscribe to bruceclays blog, much as i'd like to, because it doesn't contain the full feed. That's probably deliberate on their part, because I have to consciously visit their blog on my desktop at work, to see the whole post. The other problem with say Pulse, is how it locates the feed. I typed in the URL, and Pulse subscribed me. I assume that Pulse simply looked for the domain.com/feed URL and added that, rather than look for feeds2.feedburner.com/domain. I looked at Feedburner stats and they didn't go up for 2 days, so basically it didn't track me. Would it be as simple as using the Google URL builder to add parameters to each post in the RSS feed? Eg utm_source=feedreader, utm_medium=rss, utm_campaign=tracking. But that still wouldn't track offline users. I assume that most people are also not going to paste the Feedburner URL into their FeedReader, but would let the platform auto-detect the feed. Any suggestions?
Reporting & Analytics | | ozgeekmum1 -
What's the best way to track moble traffic / sales from a subdomain in Google Analytics?
Hello, I've been trying to figure out the best way to track traffic and sales for a subdomain mobile site and would appreciate any feedback or guidance. I've set up an advanced segment but have found that I'm not able to source traffic since the segments get disabled when trying to view top content pages. I've also setup an additional sub-profile under the main domain profile in google analytics and have set up an advanced filter using the following guildelines http://code.google.com/apis/analytics/docs/tracking/gaTrackingSite.html In the Profile Settings page, click the Add Filter link. Choose Add New Filter and provide the filter a name. Choose Custom Filter and select Advanced on the Filter type settings. Under Advanced settings: FieldA should be set to Hostname FieldB should be set to Request URI Set the values for both Field A and Field B to (.*), which is an expression that captures all characters. Set the Output To --> Constructor option to Request URI and provide $A1$B1 as the value for that choice. I'm unsure if I've set up the profile / filter correctly, the traffic stats are different than the segment. The mobile profile is also tracking all ecommerce transactions instead of just mobile. Here's the code I have in place. we're using the Traditional code Thanks, Jamie
Reporting & Analytics | | marketing_zoovy.com0