Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
Whatstuffwherebot user agent messing up Google Analytics
-
Starting yesterday, Aug 26, 2020, I noticed a new bot crawling our site with user agent whatstuffwherebot.
Google Analytics is counting these hits as human traffic, completely throwing off my numbers - yesterday, Analytics reported nearly triple my typical number of visitors.
As of now, Search Console only shows data through Aug 25 so I don't know if Search Console is also affected.
Is anybody else seeing something similar?
Does anybody know what the whatstuffwherebot bot is? I don't get any results when I search on Google or Bing.
For what it's worth, the traffic is coming from Columbus, OH, running over Amazon AWS via 278 different IP addresses so far. Also, WordFence (my WordPress security plugin) correctly identifies these hits as bot traffic.
-
Hi Martijn,
Thanks for the reply.
Yes, I've enabled bot filtering on all views except my unfiltered view. My site gets so much bot traffic that the data would be unusable without it.
As for filtering the traffic, I'm not aware of any way to retroactively filter out User-Agent unless I use GTM to create a custom dimension as described here: https://www.addsearch.com/docs/analytics/addsearch-bot-google-analytics/
Perhaps this is something I should do in case something like this ever happens again in the future.
I'm still hoping for other suggestions - particularly ways to ask the Google Analytics team to include "whatstuffwherebot" as part of their bot filtering.Best,
Akira -
Hi Akira,
Have you enabled bot filtering as a setting in Google Analytics on a view level? If not, you could always add a filter on the property or account level to filter out the traffic if it has very common patterns as you're describing. Usually, I would just stay away, from filtering out this type of traffic though if you have large amounts of traffic as otherwise you might be doing more harm to your data long-term than helping it.
Martijn.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to change domains in Google Analytics without losing the data
Hi there, We recently changed our domain from .COM to .NET so that all our subdomains from external pages matched. Right now in Google Console we have our new .NET website being tracked, but in GA we are still tracking .COM. It is also causing issues with MOZ crawling our site because of the .COM/.NET discrepancy. My question is what is the best way to change our Google Analytics from .COM to .NET without losing historical data and what considerations do we need to change before implementing this? Our team was concerned that just downloading the old data would be too vast and it we wouldn't be able to continue manipulating it dynamically in GA. Thanks!!
Reporting & Analytics | | cPanel-LLC.0 -
Will changing the property from http to https in Google Analytics affect main unfiltered view?
I set my client up with an unfiltered view in Google Analytics. This is the one with historical data going back for years, so I don't want to do anything that will affect this view. Recently, the website moved from HTTP to HTTPS. There's a setting for the property that will allow me to change the property name to https://EXAMPLE.com and change the default URL to https://EXAMPLE.com. Questions: 1. If I change the property name and the default URL, will this somehow affect my unfiltered view in a way that I'll lose historical data or data moving forward? 2. I have heard that changing the default URL to HTTPS will help me avoid a common problem others have experienced (where they lose the referrer in Google Analytics and a bunch of their sessions go to direct / other). Is this true?
Reporting & Analytics | | Kevin_P3 -
Direct traffic spam on Google Analytics: how can you identify and filter it?
One of my smaller clients noticed a huge jump in direct traffic visits last month. The bounce rate was around 97% so I'm pretty certain that most of the traffic was illegitimate. I know how to filter out spam referrals and organic keywords in Google Analytics. However I'm not sure what to do about direct traffic spam. Are there recommendations for filtering this out? Can I identify spam IP addresses?
Reporting & Analytics | | RosemaryB0 -
Can you arrange Google Analytics source/medium traffic by percentage change?
I'm doing a year to year traffic audit for a client. I would like to analyze Google Analytics source/medium traffic by percent change. Is there a way to do this? Do I have to create a custom variable? 9BH70RO
Reporting & Analytics | | VanguardCommunications0 -
Is it possible to use Google Tag Manager to pass a user’s text input into a form field to Google analytics?
Hey Everyone, I finally figured out how to use auto event tracking with Google Tag Manager, but didn't get the data I wanted. I want to see what users are typing into the search field on my site (the URL structure of my site isn't set up properly to use GA's built-in site search tracking). So, I set up the form submit event tracking in Google Tag Manager and used the following as my event tracking parameters: Category: Search Action: Search Value When I test and look in Google Analytics I just see: "search" and "search value." I wanted to see the text that I searched on my site. Not just the Action and Category of the event.... Is what I'm trying to do even possible? Do I need to set up a different event tracking parameter? Thanks everyone!
Reporting & Analytics | | DaveGuyMan0 -
Comparing % Change, Google Analytics
Hey Mozzers, Is there a simple way to compare the "% Change" in traffic when comparing two separate time periods in a single Google Analytics report? When comparing data from two separate time periods, an exported CSV doesn't include the % Change (booo!), and there's no option to sort by % Change within the GA report, essentially forcing you to scroll through all the results to pinpoint the major movers and shakers. I'm not averse to using spreadsheets to sort this data, but I'm thinking that I'd likely need a macro to make this work, something like this. However, none of the macros on that page are working (possibly because they were designed for a previous version of Analytics). All suggestions are appreciated. Thanks!
Reporting & Analytics | | dangaul0 -
Setting Up Google Analytic with Sub Folder Sites
What is the best way of setting up Google Analytic for a website that has many sub folders? The main site is example.com and it has 40 sub folder sites like example.com/uk example.com/France etc etc Would it be advised to track a single domain in Google Analytic then create filters for the sub folder sites. Filters > Include traffic from > Sub directories Also with this method is it possible to view overall incoming website stats for everything? Previous experience would be great with this thanks 🙂
Reporting & Analytics | | daracreative0 -
Google Analytics: Difference Between Goal Conversions & Goal Completions
When using Google Analytics, what is the difference between total goal conversions and total goal completions? We have many goals set up in a lead generation environment. Therefore, the only element of conversion is submitted a lead and arriving on the "Thank You" page. THose thank you pages are tagged accordingly. When we run reports though, the number of "Total Goal Conversions" and "Total Goal Completions" never match up.
Reporting & Analytics | | eMagineSEO0