Identifying Bots in Google Analytics
-
Hi there,
While you can now filter out bots and spiders in Google Analytics, I'm interested in how you identify a bots and spiders in the first place. For example, it used to be thought that Googlebot wouldn't appear in GA as it 'couldn't process Javascript' but now Google has announced new developments for its crawler with regards to interpreting javascript and CSS, this argument isn't as cut and dry.
I'm not suggesting Googlebot appears in Google Analytics, but I am saying that you can't make the case that it won't appear only because it can't interpret JavaScript.
So, I'm interested to see what metrics you use to identify a bot? For me, the mix of Users > Browser, Users > Operating System Version is still quite handy, but is it possible to identify individual bots and spiders within Google Analytics? And would Googlebot appear?
-
Hi there,
That's great - thanks. I'm aware that GA can block bots data from appearing within the settings (I think I kind of flagged this in the first sentence of my question, but probably could have been clearer), but I was a bit more interested in identifying potential bots and spiders within GA; e.g are there any noticeable tells within the browser version/operating system data that can suggest bot behaviour.
For example, if you segment a browser version, operating system and then filter only for pageviews where time on page ≤ 00:10 or less (for example) can you start to isolate bot behaviour in GA?
Edit: or are the numbers so small (due to the amount of bots that can execute javascript) that there wouldn't be much data?
-
Hi,
By default Google Analytics blocks all the well know bots and spiders that are out there and I would say the do a pretty good job. As enabling the feature to disable even more bots and spiders made less than 1% of visitors disappear with over millions of visitors.
They're able to identify bots and spiders based on the user agents so that's why they can also easily block them.
-
Hello.
I recommend reading the following articles:
http://www.lunametrics.com/blog/2014/08/07/bot-spider-filtering-google-analytics/
In short, there is a checkbox under Admin settings. > _View _panel>"View Settings." But eitherway read about it to see if it is worth it for you.
They are very recent and give a lot of insight on the subject.
Adidtionaly, is your site using wordpress or a similar CMS? If so, there are a couple of pligins that ell you when a bot has visited your site.
Thanks and hope it helps
Ricardo
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Analytics - Referral traffic being reported as Direct traffic
Hi, I have a link on my www homepage to another subdomain website. Both websites are served on https, and have independent Google Analytics properties. However, the traffic from the www site to the subdomain site is being reported as Direct, and not Referral traffic. Any ideas why? See image provided for context... RCn6HER
Reporting & Analytics | | RWesley1 -
Email traffic tracking in Google Analytics
Hello! I have a question about how to assure email traffic is properly tracked in GA > Acquisition > All Traffic > Channels. **First, some background... ** Our company (Wisconsin's largest group dental practice) is about to revamp the way we try to re-capture patients who don't have a future appointment set with us. Part of that process will include emails. Those emails will point back to our website to request an appointment. Now to the question... Is there anything special we should do to assure that links coming to the website and the resulting appointment request goal conversions track appropriately and appear under "Email" in the Default Channel Group of Analytics > Acquisition > All Traffic > Channels area of GA? For example, should we use Campaign URL Builder to establish UTM links? Thanks in advance for any feedback. Erik
Reporting & Analytics | | SmileMoreSEO0 -
Is there a way to apply the same google analytics filter to multiple properties?
I manage WordPress multiple sites for my clients. Some of these clients are SEO customers. One issue I have with the analytics reports is the occurrence of spam and ghost spam. I know how to create filters to block however there are always more and more. Is there away to export the filter and import it to all the other properties i manage? Or do they just have to be done manually every time i need to block spam?
Reporting & Analytics | | donsilvernail1 -
How to trigger a pageview using Google Analytics API upon form submission
I need to set up a goal using the G A API to trigger a pageview when someone submits a lead on my site. There's no "thank you"page currently. How do I go about it? There's no "thank you"page currently. How do I go about it? I have never used the API but am very familiar with analytics. Thanks!
Reporting & Analytics | | Majsan0 -
Google Analytics Average Position
I'm looking at Google Analytics -> Acquisition -> Search Engine Optimization -> Queries reports. I'm looking at keywords and the average position. What Google reports and what I see in a Google incognito search is different (usually my search is much lower). For example, for one search term, Google reports 5.8 average position and every time I search it is 8. My local result is 4. Anyone know why this is? I'm wondering if Google is averaging the Local results into number?
Reporting & Analytics | | CalicoKitty20000 -
Are RSS fees tracked by Google Analytics?
In both the new GA and former version, how are RSS feeds handled by Google Analytics. If not, is there an easy means by which to account for this traffic, in order to have a better picture of traffic. Thanks. Alan
Reporting & Analytics | | ahw0 -
Google Analytics underreporting
I have a site selling items, I use paypal buttons and get a report using ipn back from paypal telling me when I have sold a item. I also have a Google analytics event setup on the button, and one on the confirmation page.
Reporting & Analytics | | AlanMosley
For example in the last few days I have sold over 10 items from the website, but have only 2 events from the button and none from the confirmation page. The confirmation page depends on the user getting redirected back from PayPal so I can see that being upper reported, but there is no way you can buy without clicking the button.
I have the chrome addon from google to debug your analytics code and it fires every time.
The figures seem to report only about 20% of events over a long period of time. If it was not working at all, I could assume I have not set things up correctly but this is not the case, I get a regular flow of events but at about 205 of sales.
I am very disappointed with GA, seems a waste of time. anyone else had the same expirence?0 -
New Google Analytics Site Speed tool and excel
Hello, I was wondering if there is a good tool or method to pull the new Google Analytics Site Speed data into excel and use this document to track site speeds on a weekly basis for multiple clients? Any good articles or how-to's would be awesome!
Reporting & Analytics | | Hakkasan0