Filters for referral bots ineffective
-
I'm not sure what I'm doing wrong here. I've been trying to block spam from my client's analytics for over a month and I just can't seem to get it right. I am beyond frustrated.
Here is a list of the filters I have:
And here is referral traffic from them regardless:
Specifically this is what the filter looks like. Am I doing something wrong here?
I have also put a filter in place to block traffic from Russia, and it doesn't seem to be working 100% either.
On top of that, it seems anytime I do manage to block something, another site just pops up in its place. Is it endless?
My clients are very small local business and this is absolutely ruining their analytics. I can't even show them their traffic reports since it's absolutely meaningless with all this spam. Any help would be greatly appreciated.
-
Thank you for the quick response. Made the changes you suggested and hopefully over the next few days I'll see the effect.
-
The filter verification takes a small sample of data and sometimes in that sample there won't be any change.
Another reason for what you may get this error is because the filter was saved before the verification, so you will have to delete the filter and create it again for the verification to work. Better yet you can use the filter expression in a segment it will give you a better idea of how the filter will work since the segment is not limited to a few days.
If you try to exclude by the offender don't use referral as a filter field use Campaign Source.
For the filter based on valid hostnames, there is no need to put the domain with and without "www" clientsite.com will match both.
Hope that helps,
-
I've tried the method you suggested for one of my client's sites. I created a new filter, did "Include" -> "Hostname" and added the following:
translate.googleusercontent.com|www.clientsite.com|clientsite.com
It said that it wouldn't change any of the data when I went to verify the filter, which should not be the case. Most of the traffic is from hostnames that are not set. I tried this input as well after reading a different article on the subject:
translate.googleusercontent.com|www.clientsite.com|clientsite.com
Once again to no effect.
So I tried to tackle the referral spam itself (although for this particular site, direct traffic seems to be getting a fair amount of spam as well). I created two filters ( http://imgur.com/ShhAYqv ) that included every spam referral I've ever gotten:
adviceforum.com|social-buttons.com|semalt/.com|free-share-buttons.com|buttons-for-website.com|best-seo.com|darodar.com|semalt.com
dailyrank.net|sitevaluation.org|trafficmonetize.org|4webmasters.org|100dollars-seo.com|webmonetizer.com|event-tracking.com|.*(social-buttons.com)
And, as you can see in the picture, it says there would be no effect.
Aghhhhhh! What am I doing wrong here?
-
I have recently followed Carlos' blog posts and have to say the steps worked a treat. Hopefully the Google Analytics team are working on a way to transfer filters across different analytics accounts to make it easier to keep all sites referral traffic free.
-
Hi Kat,
If you want to exclude the spammer you should use Campaign Source as a field type, instead of predefined filter use:
- Filter type Custom > Exclude
- Filter Field > Campaign Source
- File Pattern Box enter the spammer you want to exclude. You don't need to create a different filter for each one; you can create an expression like this:
4webmasters.com|free-share-buttons.com|trafficmonetize.org|morespammers
The backslashes before the dots "." and the dashes "-" are needed because custom filters use Regular Expressions. You can keep adding up to 255 characters after that you have to create another filter
If you want to try a better solution, I recommend you to try a filter based on your hostnames.
This solution requires a little more time to set up, but it has three huge advantages over the Campaign Source solution- You will stop the spam before it hits you, adding a filter for the referral after you see it will stop it, but by the time you apply it you will have already hits of the spam.
- You will need only ONE filter to stop all ghost spam, instead of creating various sets of filters.
- Lately, some of the spammers(e.g. free-social-buttons) have been hitting GA accounts with fake direct visits along with the referral, the filter for the referral won’t stop the direct visit, on the other hand. The Valid hostname filter will stop ghost spam in any form whether it shows as a referral, keyword or direct visit.
This is what I've been using on my accounts for, the last moths and I haven't received a single hit of ghost spam the vast majority.
Here are a detailed guide and more information about all these solutions and spam in Google Analytics.
http://www.ohow.co/what-is-referrer-spam-how-stop-it-guide/
Hope it helps,
-
I've tried setting up filters in the manner you explain on your site, and yet when I go to verify it, it says that it wouldn't affect my data. I most certainly have spam data from the last seven days so I'm uncertain why it's not working.
I entered in names just as you did in your article, excluding from a campaign source:
dailyrank|sitevaluation|trafficmonetize|4webmasters|100dollars-seo|webmonetizer|event-tracking|free-social-buttons
-
You need to set the exclusion for campaign source, and not the ISP domain.
I highly recommend setting an include only filter for your own hostname.
https://www.marceldigital.com/blog/2015/06/stop-ghost-spam-in-google-analytics
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to filter blog pageviews using segments with regex
I must be overcomplicating this. I have a website with a blog structure that includes year subfolders, so: website.com/2019/blog-title, website.com/2018/blog-title, website.com/2017/blog-title, etc. I want to create a segment that I can apply in Site Content > All Pages to get quick stats for all blogs, regardless of year. (Pageviews, time on site, etc.) Unfortunately, Content Drilldown doesn't work well here as it is grouping by each year. I have tried setting up a segment using regex, but am getting a mixture of all pages, not just blog pages. I have set up the segment the following way: Advanced > Conditions > Page, Matches Regex, /2019/(.*?)/ Advanced > Conditions > Page, Matches Regex, /2018/(.*?)/ Advanced > Conditions > Page, Matches Regex, /2017/(.*?)/ Am I going about this all wrong? Is my regex wrong? Any advice is appreciated! Side note: glaringly obvious that now is the time to clean up my URLs. But I'd still love to see the data in its current URL state. Thank you!
Reporting & Analytics | | SoleGraphics0 -
Any idea why this page is an absolute magnet for bots?
This page on our client's website seems to be an absolute magnet for bots, and it's skewing our Google Analytics stats: https://cbisonline.com/us/catholic-socially-responsible-esg-investing/proxy-voting/ We already filter out lots of bots in GA, primarily through a segment we created several years ago and continue to build upon, but plenty of spam traffic still manages to slip through – mostly to the page above. Last quarter, almost all of it came from two random cities in Europe, so we're going to filter out traffic from those places. (At least for now – not an ideal solution, I know.) But I'm really wondering what drives so many bots to that page in particular. Any insights would be greatly appreciated!
Reporting & Analytics | | matt-145670 -
"Display" referrals on google analytics
We have just had 200+ visits on the 'Display' channel within 5 minutes - these referrals are from all over the world and this is very out the ordinary for our business - should we be concerned? 9cHllwW
Reporting & Analytics | | RayflexGroup0 -
Domain change from .co.uk to .com - do you add .com to the referral exclusion in GA?
Hi Mozzers, Just wanted to double check that when a website domain changes from .co.uk to .com - do you add .com to the referral exclusion in GA? Do you also keep the .co.uk version in the list too? I have updated the property settings to .com Thanks Anthony
Reporting & Analytics | | Tone_Agency0 -
Is there an efficient way to block/filter referral spam in Google Analytics for a large network of websites?
Hello, everyone - I'm looking for guidance on how to block or filter referral spam in Google Analytics. But I'm needing to block for an entire network of Wordpress websites. We have two networks which total over 2,500 websites. We are currently blocking sites we find out about via htaccess. This works, but only after we see we are getting hit with the spam. Updating 2,500+ Google Analytics accounts with filtering is not an ideal option due to the time factor and the fact that new bots coming out almost daily. We can continue the htaccess method, but does anyone have any other ideas for blocking referral spam for a large network of sites? These are the other ideas we have. 1. Blocking all traffic from Russia and China based up subnets. We know many will still get through, but it should block 50% of it, we hope.
Reporting & Analytics | | copyjack
2. Moving sites to Google Tag manager. This is a huge tasks but we have seen that sites using Tag Manager are not effected, at least for now. Other ideas are appreciated!0 -
Most of the goal conversion in Google analytics showing under referral
Hi All, I have already asked this question twice here but issue is not resolved yet. http://moz.com/community/q/all-goal-conversion-in-google-analytics-showing-under-referral http://moz.com/community/q/some-goal-conversion-in-google-analytics-showing-under-referral In Google analytics most of the goal conversion in Google analytics showing under referral from payment gateway sites (Paypal.com & epdq.co.uk). I have applied all the possible solution that suggested by Moz community experts & Google Adwords support team but it is not resolved yet and it is really weird answer to give client that I have applied all possible solution and issue is still not solved. Lastly I am sharing Cross domain tracking code here that I have used on my site. Please check once and let me know if code is correct and If not correct please help me out to correct the code or suggest any other way to fix the issue Cross domain tracking code ======================= <script type="text/javascript">(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
Reporting & Analytics | | Alick300
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('require', 'linker');
ga('linker:autoLink', ['paypal.com', 'payments.epdq.co.uk'], false, true);
ga('create', 'UA-xxxxxx', 'auto', { 'allowLinker': true});
ga('require', 'displayfeatures');
ga('send', 'pageview');</script > I have hide the UID in above code. Thanks0 -
Not Provided Filter is not tracking goals
I have setup ‘Not Provided’ filter, but it is not tracking conversions accurately. Basically the filter displays goals url’s as conversion urls & not the landing url’s. For example – I would expect home page to have maximum conversions, but its shows higher visits & no conversions. Has anyone experienced this? I have attached screenshot of the ‘Not provided’ Filter. 0L0fS3h SxzRP7w
Reporting & Analytics | | mozwoz0 -
Excluding referral traffic from a specific page Google analytics
Hi, I am trying to exclude from referrals from a particular page i.e. www.domain.com/nothispage within Google analytics, I have tried a couple variations within the advanced filter (Regex etc) section without much luck, could anyone assist ? Updated-trying to do this using a filter for the entire profile. Thanks Marc
Reporting & Analytics | | NRMA0