Identifying Bots in Google Analytics
-
Hi there,
While you can now filter out bots and spiders in Google Analytics, I'm interested in how you identify a bots and spiders in the first place. For example, it used to be thought that Googlebot wouldn't appear in GA as it 'couldn't process Javascript' but now Google has announced new developments for its crawler with regards to interpreting javascript and CSS, this argument isn't as cut and dry.
I'm not suggesting Googlebot appears in Google Analytics, but I am saying that you can't make the case that it won't appear only because it can't interpret JavaScript.
So, I'm interested to see what metrics you use to identify a bot? For me, the mix of Users > Browser, Users > Operating System Version is still quite handy, but is it possible to identify individual bots and spiders within Google Analytics? And would Googlebot appear?
-
Hi there,
That's great - thanks. I'm aware that GA can block bots data from appearing within the settings (I think I kind of flagged this in the first sentence of my question, but probably could have been clearer), but I was a bit more interested in identifying potential bots and spiders within GA; e.g are there any noticeable tells within the browser version/operating system data that can suggest bot behaviour.
For example, if you segment a browser version, operating system and then filter only for pageviews where time on page ≤ 00:10 or less (for example) can you start to isolate bot behaviour in GA?
Edit: or are the numbers so small (due to the amount of bots that can execute javascript) that there wouldn't be much data?
-
Hi,
By default Google Analytics blocks all the well know bots and spiders that are out there and I would say the do a pretty good job. As enabling the feature to disable even more bots and spiders made less than 1% of visitors disappear with over millions of visitors.
They're able to identify bots and spiders based on the user agents so that's why they can also easily block them.
-
Hello.
I recommend reading the following articles:
http://www.lunametrics.com/blog/2014/08/07/bot-spider-filtering-google-analytics/
In short, there is a checkbox under Admin settings. > _View _panel>"View Settings." But eitherway read about it to see if it is worth it for you.
They are very recent and give a lot of insight on the subject.
Adidtionaly, is your site using wordpress or a similar CMS? If so, there are a couple of pligins that ell you when a bot has visited your site.
Thanks and hope it helps
Ricardo
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Analytic - Avg load Time - Page Timing Sometimes Graph goes suddenly up why?
Hello All, I have attached the avg load time screenshot for my ecommerce site for 1 month. Screenshot is for Mobile site. If you check the graph then only one day graph gone very high. Now again I have attached that particular day graph too. My query is why graph gone high 1) Is it my site not performed well on particular one day on particular device for single visitor? because for desktop and tablet graph showing normal 2) so here if site restart or down then it can be a problem for desktop, tablet too right? Can anyone give me any clue? Z4hX1 IanuP
Reporting & Analytics | | pragnesh96390 -
Discrepancy between FB PPC and Google Analytics
This question was answered in 2011 here: http://moz.com/community/q/facebook-ppc-number-of-clicks-according-to-fb-different-than-visits-in-analytics. Wanted to post it to the community to see if anyone had any new thoughts in the last 3 years. I have been running campaigns on Facebook and seeing dramatic discrepancies between Facebook and GA's numbers. For example, I ran a Facebook ad campaign for a chiropractor where FB shows 35 clicks to the website, but GA only shows 2! An attorney ran a Facbook promotion, got 4 clients who actually filled out a questionnaire online, but GA only showed 2 visitors exiting off the form completion page. Is this because the users did not have JS/cookies enabled? Something else? What is the recommended work around? Tracking URL?
Reporting & Analytics | | aj6130 -
Where does the organic keyword information come from in Google Analytics?
I know that Google switch to all encrypted search, but I still show some keywords. Are those keywords that slipped through? Or are they all from Yahoo/Bing?
Reporting & Analytics | | EcommerceSite0 -
Google analytics vs Webmaster tools data
Hi Which is more accurate WT or GA data ? Since GA reporting a KW (thats very recently fallen from page 1 to 3 hence looking into data to find the cause) in the Organic part of Search tab as having generated just 1 visitor over a month (hence presuming fall could be due to low visits from a page 1 result) whilst under Search Engine Optimisation tab (data sourced from WT i think) its reporting 5 click thrus from 150 impressions over same period resulting in a quite good 3.33% CTR (hence wouldn't expect to be the cause of a fall) and what i would have thought GA would report as 5 visits instead of the 1 they do report !? The reason im looking for answer in the data is because no on-page has changed and still scoring a grade A and off page metrics have all improved across the board (apart from small drop in majseo's Trust Flow) such as increased links, RD, Citation Flow, Ref Subnets etc etc etc Cheers Dan
Reporting & Analytics | | Dan-Lawrence0 -
Impressions in Google Analytics
I am trying to compare SERP impressions in Google Analtyics between two different time periods. I want to compare the last two months, with the previous two months. Now this works fine when I go to Traffic Sources > SEO > Queries. Our analytics has been set up since early last year, so I cannot understand why, for a couple of weeks at the start of the previous two months, it is showing that I have less than 10 impressions per day, then in one day, it jumps to 22,000 impressions, and starts to show 'real' information after that... Very frustrating when I am trying to show how effective my work has been. Can anyone shed any light on this?
Reporting & Analytics | | MirandaP0 -
Multiple Domain Tracking with One Google Analytics Account
I am working on multiple online retail stores as follow. Right now, I have tracking facility with multiple account. Attached image can give you more idea about it. http://www.vistastores.com/
Reporting & Analytics | | CommercePundit
http://www.lampslightingandmore.com/
http://www.vistapatioumbrellas.com/ I don't want to track my statistics with multiple account. Because, it may take too much time behind Goal set up as well as analysis. Can I track all data under one account for all online retail stores? 6464019053_bda32e9fe0_b.jpg0 -
Google analytics advanced segments
Ok, I need help with a simple (although, for some reason, I'm having trouble with it) advanced segment. Dilemma**:** All of our techs have a backend cookie that they use to log into our website. I want a way toexclude all visites where the landing page contained this in the url:/backend/cookie.php?username= Advanced Segment: We have a lot of techs and each one of them has a different "username=example". So how can I set up an advanced segment where it will exclude any visit where the visitor came in one a landing page containing /backend/cookie.php?username=
Reporting & Analytics | | NerdsOnCall0 -
Do Google Analytics filters affect link building?
We recently made a few link wheels for specific product pages. We've been having great results with all of the wheels except for one. The one we are having issues with is the only link that we were using a Google Analytics filter on; it looks like this http://domain.com/page.htm?zSource=Specific Keyword%tracking My question is does Google ignore links that are obviously utilizing their analyitcs custom filters? We're doing some more testing to try to find out if it truly is the link or if the wheel is a bad one. There are so many things that could go wrong but with so many of our link wheels working well, I wonder if the filters are what is causing such results.
Reporting & Analytics | | MichealGooden0