Identifying Bots in Google Analytics
-
Hi there,
While you can now filter out bots and spiders in Google Analytics, I'm interested in how you identify a bots and spiders in the first place. For example, it used to be thought that Googlebot wouldn't appear in GA as it 'couldn't process Javascript' but now Google has announced new developments for its crawler with regards to interpreting javascript and CSS, this argument isn't as cut and dry.
I'm not suggesting Googlebot appears in Google Analytics, but I am saying that you can't make the case that it won't appear only because it can't interpret JavaScript.
So, I'm interested to see what metrics you use to identify a bot? For me, the mix of Users > Browser, Users > Operating System Version is still quite handy, but is it possible to identify individual bots and spiders within Google Analytics? And would Googlebot appear?
-
Hi there,
That's great - thanks. I'm aware that GA can block bots data from appearing within the settings (I think I kind of flagged this in the first sentence of my question, but probably could have been clearer), but I was a bit more interested in identifying potential bots and spiders within GA; e.g are there any noticeable tells within the browser version/operating system data that can suggest bot behaviour.
For example, if you segment a browser version, operating system and then filter only for pageviews where time on page ≤ 00:10 or less (for example) can you start to isolate bot behaviour in GA?
Edit: or are the numbers so small (due to the amount of bots that can execute javascript) that there wouldn't be much data?
-
Hi,
By default Google Analytics blocks all the well know bots and spiders that are out there and I would say the do a pretty good job. As enabling the feature to disable even more bots and spiders made less than 1% of visitors disappear with over millions of visitors.
They're able to identify bots and spiders based on the user agents so that's why they can also easily block them.
-
Hello.
I recommend reading the following articles:
http://www.lunametrics.com/blog/2014/08/07/bot-spider-filtering-google-analytics/
In short, there is a checkbox under Admin settings. > _View _panel>"View Settings." But eitherway read about it to see if it is worth it for you.
They are very recent and give a lot of insight on the subject.
Adidtionaly, is your site using wordpress or a similar CMS? If so, there are a couple of pligins that ell you when a bot has visited your site.
Thanks and hope it helps
Ricardo
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Does referral exclusions remove the ecommerce value from Google Analytics?
Quick question on Referral exclusions. We have a client that just rebuilt their Magento website, referral data from Paypal was never an issue, the site properly attributed to source. New web devs finished the site and all purchases through Google Analytics from PayPal Is now showing as referral traffic. Doing research on the issue the suggestion was to create a referral exclusion in GA from PayPal. My question on that, will the ecommerce value default to the previous referral or will it delete the e-commerce value completely from what is gathered?
Reporting & Analytics | | BCutrer0 -
How to trigger a pageview using Google Analytics API upon form submission
I need to set up a goal using the G A API to trigger a pageview when someone submits a lead on my site. There's no "thank you"page currently. How do I go about it? There's no "thank you"page currently. How do I go about it? I have never used the API but am very familiar with analytics. Thanks!
Reporting & Analytics | | Majsan0 -
How to exclude IP from Google Analytic?
Hello Guys, I am admin of google analytic, i am trying to exclude traffic in google analytic from admin, view section then filter but not there is no button of save? Can anyone please help me? PFA of image. zOkvX1n.jpg
Reporting & Analytics | | devdan0 -
Determining Exact Reverse Path in Google Analytics
Within Google Analytics does anyone have suggestions for how to determine the exact reverse path for a product purchased? The goal funnel does not provide in a detailed way, especially if a step is taken out of the exact funnel provided. If this information cannot be obtained through Analytics is there another tool we can use to get the exact path?
Reporting & Analytics | | bozzie3110 -
Google Analytics Organic Search Keywords Suddenly Displaying FulL Urls
In my Google Analytics, the top keywords for Organic Search are suddenyl displaying full URLs. For example, now the third and fourth keywords are http://www.domain.com/highly-specific-URL. These have all started recently around the same day, July 12th. I've checked back, and we've made no internal changes to the site around that time that could affect this. Any thoughts on this? Thanks! P.S. It might be related to rich snippets, but I cannot tell at this point.
Reporting & Analytics | | 10SL0 -
Testing IP Exclusion Filters in Google Analytics
(I hope I haven't missed something simple 😉 I have add IP exclusion filters in Google Analytics. What I haven't been able to determine is how to test to see if it is working. I'd imagine there would be a dip in traffic but if a site is low volume it might not even register. Thanks!
Reporting & Analytics | | AUDigitalMarketing0 -
SEOMoz crawls skewing Avg Visit Duration in Google Analytics
Hello, We are a UK based company. Our Google Analytics account is showing a rise in Avg Visit Duration for 'Direct traffic' since we started using SEOMoz. Are any other users experiencing this issue? We tracked it down to City Seattle, and when doing an Advanced filter by removing Seattle, the results are normal again. What does SEOMoz or other users recommend we do besides continuously using advanced filters? We have enquired about excluding Roger's IP address, but have been told that Roger uses the Amazon cloud, so the IP is not static. See attachment for screenshot of our Google Analytics account of Avg Visit Duration since we began with SEOMoz. Rich Talbot 718g3.gif
Reporting & Analytics | | STL1 -
Google Analytics
Hello In Google analytics you can obtain the number of visits as a result of non-paid search. You can also set up custom reports to find the number of organic searches. The numbers are different, so what is the obvious difference between these two metrics that I'm missing. Thanks in advance for any assistance. Neil
Reporting & Analytics | | mccormackmorrison0