Best way to block spambots in htaccess
-
I would like to block Russian Federation, China and Ukraine spam as well as semalt and buttonsforwebsite. I have come up with the following code, what do you think?
For the countries:
BLOCK COUNTRY DOMAINS
RewriteCond %{HTTP_REFERER} .(ru|cn|ua)(/|$) [NC]
RewriteRule .* - [F]And for buttons-for-website.com and semalt-semalt.com:
BLOCK REFERERS
RewriteCond %{HTTP_REFERER} (semalt|buttons) [NC]
RewriteRule .* - [F]or should it be:
BLOCK USER AGENTS
RewriteCond %{HTTP_USER_AGENT} (semalt|buttons) [NC]
RewriteRule .* - [F]Could I add (semalt|buttons|o-o-6-o-o|bestwebsitesawards|humanorightswatch) or is that too many?
-
Hi
I think you're on the right track.
A very good blog post by Jared Gardner has recently addressed this question on the Moz Blog.
Hope it helps,
Don
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
PDF best practices: to get them indexed or not? Do they pass SEO value to the site?
All PDFs have landing pages, and the pages are already indexed. If we allow the PDFs to get indexed, then they'd be downloadable directly from google's results page and we would not get GA events. The PDFs info would somewhat overlap with the landing pages info. Also, if we ever need to move content, we'd now have to redirects the links to the PDFs. What are best practices in this area? To index or not? What do you / your clients do and why? Would a PDF indexed by google and downloaded directly via a link in the SER page pass SEO juice to the domain? What if it's on a subdomain, like when hosted by Pardot? (www1.example.com)
Reporting & Analytics | | hlwebdev1 -
Blocking Standard pages with Robots.txt (t&c's, shipping policy, pricing & privacy policies etc)
Hi I've just had best practice site migration completed for my old e-commerce store into a Shopify environment and I see in GSC that it's reporting my standard pages as blocked by robots.txt, such as these below examples. Surely I don't want these blocked ? is that likely due to my migrators or s defaults setting with Shopify does anyone know? : t&c's shipping policy pricing policy privacy policy etc So in summary: Shall I unblock these? What caused it Shopify default settings or more likely my migration team? All Best Dan
Reporting & Analytics | | Dan-Lawrence0 -
Should I block Ukraine, China and Russian Federation for better SEO?
My statistics show that a large percentage of hits to my site are from Ukraine, China and the Russian Federation. Now I know for a fact that most of my demographic is from the US with a bit of Western Europe thrown in. So obviously these are spam bots. Also I have many linked pages with the .ru extension that are spammy. Should I block Ukraine, China and the Russian Federation in my htaccess? I would think bounce rate is very high with spam bots so would that improve my SEO? Then of course there is the extremely annoying buttons-for-website.com and semalt-semalt.com!
Reporting & Analytics | | ijb0 -
Adding Something to htaccess File
When I did a google search for site.kisswedding.com (my website) I noticed that google is indexing all of the https versions of my site. First of all, I don't get it because I don't have an SSL certificate. Then, last night I did what my host (bluehost) told me to do. I added the below to my htaccess file. Below rule because google is indexing https version of site - https://my.bluehost.com/cgi/help/758RewriteEngine OnRewriteCond %{HTTP_HOST} ^kisswedding.com$ [OR]RewriteCond %{HTTP_HOST} ^kisswedding.com$RewriteCond %{SERVER_PORT} ^443$RewriteRule ^(.*)$ http://www.kisswedding.com [R=301,L] Tonight I when I did a google search for site:kisswedding.com all of those https pages were being redirected to my home page - not the actually page they're supposed to be redirecting to. I went back to Bluehost and they said and 301 redirect shouldn't work because I don't have an SSL certificate. BUT, I figure since it's sorta working I just need to add something to that htaccess rule to make sure it's redirected to the right page. Someone in the google webmaster tools forums told me to do below but I don't really get it? _"to 301 redirect from /~kisswedd/ to the proper root folder you can put this in the root folder .htaccess file as well:_Redirect 301 /~kisswedd/ http://www.kisswedding.com/" Any help/advice would be HUGELY appreciated. I'm a bit at a loss.
Reporting & Analytics | | annasus0 -
Using Clients GA account for their over all website to Optimize Their Blog? or is there a better way to compare apples to apples
I would like to optimize my clients blog. If I am using their GA account for their over all website but put in the sub folder for SEOMOz, will it detect that I only want to optimize the blog ex. http://xxx.com/blog or will I be seeing the GA for the entire site on SEOMOZ?
Reporting & Analytics | | CliffordC0 -
Best SEO Analysis Tool
I'm looking for a tool that will analyze a webpage for a specific keyword phrase and deliver good recommendations as to what I can do (on page and off page) to increase the Google ranking for that keyword phrase. I'm trying IBP now, but so far the results haven't been good. Any suggestions?
Reporting & Analytics | | urbanscooters0 -
Whats the best way to separate Google Shopping from regular organic traffic in analytics?
whats the best way to separate Google Shopping from regular organic traffic in analytics?
Reporting & Analytics | | DavidKonigsberg0 -
Best Practices For Google Analytics
Does anyone have any recommendations or resources for best practices in using Google Analytics. Specifically, as they relate to SEO and managing and monitoring several websites (i.e. registering/managing multiple accounts vs. master account, etc).
Reporting & Analytics | | Gyi0