Identifying Bots in Google Analytics
-
Hi there,
While you can now filter out bots and spiders in Google Analytics, I'm interested in how you identify a bots and spiders in the first place. For example, it used to be thought that Googlebot wouldn't appear in GA as it 'couldn't process Javascript' but now Google has announced new developments for its crawler with regards to interpreting javascript and CSS, this argument isn't as cut and dry.
I'm not suggesting Googlebot appears in Google Analytics, but I am saying that you can't make the case that it won't appear only because it can't interpret JavaScript.
So, I'm interested to see what metrics you use to identify a bot? For me, the mix of Users > Browser, Users > Operating System Version is still quite handy, but is it possible to identify individual bots and spiders within Google Analytics? And would Googlebot appear?
-
Hi there,
That's great - thanks. I'm aware that GA can block bots data from appearing within the settings (I think I kind of flagged this in the first sentence of my question, but probably could have been clearer), but I was a bit more interested in identifying potential bots and spiders within GA; e.g are there any noticeable tells within the browser version/operating system data that can suggest bot behaviour.
For example, if you segment a browser version, operating system and then filter only for pageviews where time on page ≤ 00:10 or less (for example) can you start to isolate bot behaviour in GA?
Edit: or are the numbers so small (due to the amount of bots that can execute javascript) that there wouldn't be much data?
-
Hi,
By default Google Analytics blocks all the well know bots and spiders that are out there and I would say the do a pretty good job. As enabling the feature to disable even more bots and spiders made less than 1% of visitors disappear with over millions of visitors.
They're able to identify bots and spiders based on the user agents so that's why they can also easily block them.
-
Hello.
I recommend reading the following articles:
http://www.lunametrics.com/blog/2014/08/07/bot-spider-filtering-google-analytics/
In short, there is a checkbox under Admin settings. > _View _panel>"View Settings." But eitherway read about it to see if it is worth it for you.
They are very recent and give a lot of insight on the subject.
Adidtionaly, is your site using wordpress or a similar CMS? If so, there are a couple of pligins that ell you when a bot has visited your site.
Thanks and hope it helps
Ricardo
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
The best way to track internal links in Google Analytics
Hi there, we are a retail business and we have invested in quality editorial content which sits in our Blog at ourwebsite.co.uk/blog/ The Blog links to the main site (an online store) and I want to track the 'value' of the blog by how many clicks the blog content generates back to the main store. At the moment we're using this code on the end of every link in the Blog: ?utm_source=Blog&utm_medium=Widget&utm_campaign=FromBlog Does this affect SEO and is there a better way of doing it? Thanks.
Reporting & Analytics | | Bee1590 -
Query on google analytic benchmarking report
Hi All, First I select My Industry Vertical - ABCD then I select Region - US ( all region) then size by daily session - 10000-99999 finally reports appears "Blank" but when I decrease daily session from 10000-99999 to 1000-4999 or less then report is perfect. So what does it mean? My Avg daily session is 70k to 80k. So how to analysis benchmarking in this case? Thanks!
Reporting & Analytics | | pragnesh96390 -
Does Google Analytics track conversions in real time?
Looking at the section: Conversions > Goals > Overview - does Google Analytics provide real time / same day metrics for this conversion data or does it take 24 hrs?
Reporting & Analytics | | Rich_Coffman0 -
Google analytics vs Webmaster tools data
Hi Which is more accurate WT or GA data ? Since GA reporting a KW (thats very recently fallen from page 1 to 3 hence looking into data to find the cause) in the Organic part of Search tab as having generated just 1 visitor over a month (hence presuming fall could be due to low visits from a page 1 result) whilst under Search Engine Optimisation tab (data sourced from WT i think) its reporting 5 click thrus from 150 impressions over same period resulting in a quite good 3.33% CTR (hence wouldn't expect to be the cause of a fall) and what i would have thought GA would report as 5 visits instead of the 1 they do report !? The reason im looking for answer in the data is because no on-page has changed and still scoring a grade A and off page metrics have all improved across the board (apart from small drop in majseo's Trust Flow) such as increased links, RD, Citation Flow, Ref Subnets etc etc etc Cheers Dan
Reporting & Analytics | | Dan-Lawrence0 -
Google Analytics - Tracking a Goal from 1 Domain to Another
Hi there, been combing this page for some answers - https://developers.google.com/analytics/devguides/collection/gajs/gaTrackingSite but can't seem to figure it out. Scenario - Site A Site B Currently have 2 different GA accounts set up on each. Is it possible to Track a Goal that happens on Site B, that comes from Site A - In Site A Reports? So say, Site A has a Newsletter, directing them to Site A but promoting Site B - i'd like to track the effectiveness of the newsletter. Site A is running a white label of Site B service. Thoughts? Cheers
Reporting & Analytics | | IsHot0 -
Set up Google Analytics by product category
Does anyone know if it's possible to set up Google Analytics data by product category? We sell roughly twenty product categories on our ecommerce site. We can look at our analytics performance for the site overall and drill down to specific pages, but what we really want to see is our performance by product category. Our product categories can include up to 5000 products. Any pointers on how to do this?
Reporting & Analytics | | ironpac0 -
Custom Variables to track Vimeo plays on website with Google Analytics?
Hello Everyone, I'm trying to track how many times a Vimeo video is played on my site via GA. Does any of you have any knowledge of how can this be achieved? I've read the documentation and came up with this: After the iframe embed i insert this: Of course the GA is loaded in the header. Does not work, at least i cant see anything in analytics. I have set up the segment as per the attached image. Thanks in advance! Alex E6XnO.png
Reporting & Analytics | | pwpaneuro0 -
Google Analytics/ Contact 7 Plugin
I have a site that uses the contact 7 plugin for its contact form sitewide in a sidebar and on a contact us page. I've set up a goal in analytics and tracked using the "on_sent_ok" hook but the number of goals GA is creating is far outweighing the number of times the form is actually completed. I figured my initial mistake was to use the contact-us page as the page tracker variable for the goal but wondered if it was counting all form completions AND anyone who went to the contact page. So I amended it to a contact-us/thank-you page that no site visitor could navigate to, and since I have no goals completed, even though we're getting enquiries through the form. Am I being dumb here and missing something quite simple?
Reporting & Analytics | | PerchDigital0