Stripping Out Referral Spam From Past Reports
-
Hi,
I'm looking to confirm the best approach for retroactively stripping away referral spam (free buttons, SEMalt, etc.). Now to be clear, I already have filters in place to ignore them from current stats, so moving forward I'm fine. However, I'd love to go back and check untainted stats.
I've setup segments using a regex to strip the root words away and it seems to be working. I have a regex setup to strip out things like: social-buttons|seoanalyses|copyrightclaims|classifiedads|jobsense|free-share-buttons|e-buyeasy|acrobats.hol|cheap-online|amezon|search-help|qut-smoking and so forth.
I've been going through my referral data, noticing obvious spam, and adding their domains to my segment. Is this the optimal way for me to get a clear, untainted view of my past stats?
-
Sweet, glad to hear our filters will suffice. Thanks for the input, Daniel.
-
Hey, no worries and you're right that your filters should block them as well. Using .htaccess would be just an additional defense mechanism but may not be necessary.
-
Hi Daniel,
Thanks again for the response. What would be the difference in Analytics data between my filters and going straight to .htaccess? If the data is the same, is there an additional benefit to .htaccess?
For regular users, I'd suspect less bandwidth since they can't load my domain, but I don't think these bots actually load the page or visit.
-
I would use your .htaccess file to block them with the following code (this would for example block referrals from semalt.com and semalt.com subdomains):
RewriteEngine On
Options +FollowSymlinks
RewriteCond %{HTTP_REFERER} ^https?://([^.]+.)*semalt.com\ [NC,OR]
RewriteRule .* – [F]
You can also use .htaccess to block IP addresses associated with the spammy sources.
edit: just saw your edit but hope this helps nevertheless!
-
Hi Daniel,
Thanks for the additional tips. I do have the bot filtering feature enabled as another point of protection. I checked my referral exclusion list and apparently set this up about a year ago for the initial wave of referral bots I noticed. I didn't know it added them to direct.
The majority of my spam referral hosts have been added to regular filters. I think with the combination of my retroactive approach and new filters, I should have reliable data going forward.
-
Hi there,
You’re on the right track and the best way to retroactively remove spammy sources is through report filters and advanced segments.
A couple other notes:
- A good way to spot spammy referrers is to sort by bounce rate and eliminate any with 100% bounce and over 10 sessions.
- Avoid using the “referral exclusion list” since this will just count spam traffic as direct traffic instead.
- You should also enable the GA ‘bot filtering’ feature under ‘Reporting view settings’ as seen here
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Solved Is there a way to remove the Moz branding from automated PDF reports that are emailed to clients?
Is there a way to remove the Moz branding from automated PDF reports that are emailed to clients?
Reporting & Analytics | | ArttiaCreative0 -
Google My Business app search report
What is the deal with the search reports in the Google My Business app? I downloaded this app so prospective customers could message my business, and when I look at the search reports on the app, the results seem nonsensical. According to google analytics my business receives pretty steady traffic every day. Why does the report say that I receive zero visitors one day and 400 the next? (See the screenshot below) 4yLRCHG
Reporting & Analytics | | RandyHT1 -
Referral Data Q's
1. We recently ran a promotion on both FB and Reddit, which is https, linking to our non-https site. We utilized UTM links to our landing page. Our GA campaign data returned extremely low hits in comparison to what we actually received (and recorded via FB/Reddit dashboard). Obviously our Direct traffic spiked during these times, caused by a secure to nonsecure referral, I'm sure. I'm also noticing a spike in referral traffic from lm.facebook.com that correlates to the ad times. Does this mean Facebook's link shim is stripping away my UTM data? My question is why we receive SOME properly UTM-tagged referral traffic in our campaigns? What's allowing some of it to go through? 2. I've tagged our email signature links with UTM as well, hoping to clean up some of our Direct traffic. I understand that external clients like Outlook and Thunderbird likely won't pass referral data, but do hosted clients like Gmail, Yahoo, and such? And if so, would the https to http difference obstruct this again? I'd love some insight onto these questions, especially if I'm off the mark with a few of my assumptions there.
Reporting & Analytics | | kirmeliux0 -
Google Analytics shows most referrers as "Direct" -- What are some better tools?
Very often Google Analytics will show 50-90% of our referrers as (direct) which is not very helpful. Are there other tools out there that will provide a clearer breakdown of what other websites are sending us our traffic? Specifically, I want to be able to be able to tell who are the top traffic referrers to my top performing pages on my site for the last 30 days. (I want to be able to study this on a per-page basis.) Thanks in advance!
Reporting & Analytics | | Brand_Psychic0 -
Google Maps not passing referral data
Google Maps is not passing referral data (URLs, not KWs). Google+ Local is referring, but nothing from maps. Maps referrals appear to be coming across as direct. Any ideas? We haven't found anything online, one of the guys at the office documented what we did find, using Chrome's debugger - http://manofactionmetrics.com/2012/11/02/google-maps-not-passing-any-referral-data/
Reporting & Analytics | | Danieljacobree0 -
Why did my site go from 1,000 Impressions to 0 impressions over the past couple days
so we have been doing a lot of reconstruction our website the past couple days with our navigation and moving stuff around. Now bare with me because i am a noob to SEO and i am still learning. But we went from a steady 800-1000 impressions everyday to a 0 since the first. you can check out our site here: www.hidguy.net Not sure if its because we are moving stuff around or what. But we can notice that our conversion rate has dropped and we have had traffic but no conversions and no orders. Is what we are doing bad or is this normal?
Reporting & Analytics | | Dante130 -
A lot of traffic to one page from Google referral
We recently received a lot of traffic to one page from
Reporting & Analytics | | underthesun808
google.com referral. When I look in analytics it reports that the traffic is
coming from /url that’s not real helpful. Is there a way to get more specific
information as to what the referring url was?0 -
Google Analytics help - exporting reports
Hey mozzers, A little help if you will. I am in traffic sources > sources > search > organic I am view keywords and comparing 1 month with another. Now I see what 1 keyword sends 1st month, 2nd month and % Change. I want to export this data to an .xls file. When I do, I end up with a file that shows what 1 keyword sends 1st month and 2nd month, but no % Change! I really need to include this in exported document.... Summary: Analytics hates me. 🙂
Reporting & Analytics | | MirandaP0