Identifying Bots in Google Analytics
-
Hi there,
While you can now filter out bots and spiders in Google Analytics, I'm interested in how you identify a bots and spiders in the first place. For example, it used to be thought that Googlebot wouldn't appear in GA as it 'couldn't process Javascript' but now Google has announced new developments for its crawler with regards to interpreting javascript and CSS, this argument isn't as cut and dry.
I'm not suggesting Googlebot appears in Google Analytics, but I am saying that you can't make the case that it won't appear only because it can't interpret JavaScript.
So, I'm interested to see what metrics you use to identify a bot? For me, the mix of Users > Browser, Users > Operating System Version is still quite handy, but is it possible to identify individual bots and spiders within Google Analytics? And would Googlebot appear?
-
Hi there,
That's great - thanks. I'm aware that GA can block bots data from appearing within the settings (I think I kind of flagged this in the first sentence of my question, but probably could have been clearer), but I was a bit more interested in identifying potential bots and spiders within GA; e.g are there any noticeable tells within the browser version/operating system data that can suggest bot behaviour.
For example, if you segment a browser version, operating system and then filter only for pageviews where time on page ≤ 00:10 or less (for example) can you start to isolate bot behaviour in GA?
Edit: or are the numbers so small (due to the amount of bots that can execute javascript) that there wouldn't be much data?
-
Hi,
By default Google Analytics blocks all the well know bots and spiders that are out there and I would say the do a pretty good job. As enabling the feature to disable even more bots and spiders made less than 1% of visitors disappear with over millions of visitors.
They're able to identify bots and spiders based on the user agents so that's why they can also easily block them.
-
Hello.
I recommend reading the following articles:
http://www.lunametrics.com/blog/2014/08/07/bot-spider-filtering-google-analytics/
In short, there is a checkbox under Admin settings. > _View _panel>"View Settings." But eitherway read about it to see if it is worth it for you.
They are very recent and give a lot of insight on the subject.
Adidtionaly, is your site using wordpress or a similar CMS? If so, there are a couple of pligins that ell you when a bot has visited your site.
Thanks and hope it helps
Ricardo
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Why No Goal is recorded in Google Analytics
Hello, I am not sure if i made an error. Can someone please point out. On our sales form, when a user submits the form, the URL displayed is - https://x-y.com/thank-you-2/ I created a Goal like this - 1 ) Under Goal set up, i choose Template option 2 ) Under Goal description, i choose Type > Destination 3 ) and finally, in Goal details field, destination equals to /thank-you-2/ But, no Goal is being tracked. In the first step, should i have selected 'Custom' instead of 'Template' Thanks
Reporting & Analytics | | Johnroger0 -
Keeping Google Analytics Data when Moving to Subdomain
Hey All, Against my objections a client has decided to move an existing site into a subdomain while putting up a new site on the main domain. My question revolves around Google Analytics, how do I make sure that I don't lose historical data on the domain before it moves to a subdomain? We're going to be doing a redesign of the old site and I need to keep the historical data so I can prioritize content. What do I need to do? Or will Google analytics recognize the URL's and still attribute the data to those URL's or will I have a separate set of data based on the new URL (with the subdomain). Any insight would be appreciated! Thanks!
Reporting & Analytics | | EvansHunt0 -
Google Analytics - Still Seeing Keyword Data
Hi, I hope you can answer this question for me, obviously I'm aware of the changes regarding "not provided" in analytics however I am still getting keyword referral data in analytics (not much of course!) in the section Acquisition>Keyword>Organic Can anybody explain why I am still seeing organic keyword data even up until yesterday for the odd term? I cannot find an answer anywhere!!! Many thanks
Reporting & Analytics | | splendidapple1 -
Using Filters On Google Analytics
Hi Guys, I am In need of some technical help with regards to Google analytics. One of my clients has a site providing HR Software on SaaS basis. The users login via the site for the various HR Tools. Obviously this is skewing the analytics data and showing 10,000+ extra visitors that actually don't exist. Now I understand i can add filters and exclude certain pages for instance the login page, however, some users will visit the home page then click login. How can I exclude them just to show new visitors to the site rather than users that are just trying to login. Hope this makes sense Neil
Reporting & Analytics | | nezona0 -
Google Analytics and DNS change
Our new alumni application is going be tested at domain uva.imodules.com . We are going to collect traffic data with a Google analytics account number UA-884652-XX. So going to uva.imodules.com/myPage.html would send its data to Google Analytics with that account number. Then when it is ready for production we are going to just change the domain name of the application and switch the DNS over to dardencommunity.darden.virginia.edu . So going to dardencommunity.darden.virginia.edu /myPage.html would send its data to Google Analtics with that SAME account number. Aside from having the testing domain data in the same profile are there any other issues/problems we may run into?
Reporting & Analytics | | Darden0 -
Google Analytics traffic hijacking?
Ran into something interesting a week ago - the same Google Analytics code was installed on two different sites by accident. The account was reporting traffic from both domains. Haven't found a definitive answer on how to stop this yet if it were to be used maliciously?
Reporting & Analytics | | khemistry0 -
Set up Google Analytics by product category
Does anyone know if it's possible to set up Google Analytics data by product category? We sell roughly twenty product categories on our ecommerce site. We can look at our analytics performance for the site overall and drill down to specific pages, but what we really want to see is our performance by product category. Our product categories can include up to 5000 products. Any pointers on how to do this?
Reporting & Analytics | | ironpac0 -
Track image searches on google analytics
How can i track image searches on google analytics? To filter the refferers by image.google or /imgres does not work in new google analytics (and brings an impossibly small number on old GA). also, this would not cover local google image refferers, that don't come from image.google.com. anyone knows how to find image search sources? i haven't found anything helpful in google's documentation or forums. thanks!
Reporting & Analytics | | zeepartner0