KW Explorer is Working to Disambiguate Keywords Google Merges Together
-
Hey gang,
Russ Jones from Moz has been doing a ton of heavy lifting work to try to get around the new problem posed by Google AdWords recent change to merged-keyword volume data. But, we're fighting back against this obfuscation in Keyword Explorer. I'm sharing two emails (slightly edited) from Russ about what we're doing here:
Introduction to the Problem:
Google Adwords Keyword Planner is the primary source for keyword search volume (how often a keyword is searched monthly on Google) for much of the search marketing industry. While Google has grouped together highly-similar terms for a while (especially misspellings), in June of 2016 they dramatically increased this keyword-grouping. This means similar phrases like "keyword rank", "keyword ranking" and "keyword rankings" would all be reported as having the same, combined search volume, rather than their individual search volumes. If you were to take Google's numbers at face value, you might think there are 3,000 searches per month for these 3 terms, when in reality there is only 1,000, divided amongst the 3 terms.
How we are addressing it:
Moz's Keyword Explorer uses a blend of data sources, not just Keyword Planner, to build our volume metrics. This gives us a distinct advantage in that we can adjust the volume of words that deviate dramatically in one data set verses another. Take for example the phrases "keyword rank", "keyword ranking", and "keyword rankings". While Google Keyword Planner might report all of these as having 1,000 searches per month, Moz Keyword Explorer can detect that these numbers are significantly higher than what our models would predict given our other data sets. We can then adjust the volume accordingly. Moreover, given our huge keyword data set, we can also identify grouped phrases (like these 3) and divide the volume proportionally to what we see in our other data sets. Thus, we address the grouping problem from multiple directions.
Here's email #2 from Russ, detailing more of how we're attacking this:
I have been working pretty much non-stop on this keyword volume disambiguation problem (finding the real search volume of individual keywords when Google clumps several together). I think I have settled on a pretty good solution and am working on getting it all in. For example...
Google Keyword Volume for the phrases "briefcase for women" and "briefcases for women" are both at 3600 because they have been lumped together. My disambiguation script says the singular (briefcase for women) should be 2731 and the plural should be 869. Google Trends roughly agrees with this, showing that the singular is searched more than 2x the plural: https://www.google.com/trends/explore#q=briefcase%20for%20women%2C%20briefcases%20for%20women&cmpt=q&tz=Etc%2FGMT%2B4
Basically, Keyword Explorer should already be providing some more accurate/segmented numbers than AdWords, and in the future, we'll get even better thanks to our clickstream data and our evolving models.
Any questions, let us know!
-
Thank you, Russ!
Google confirmed me that this change is due to an update, it won't be rolled back and there's no way to fetch data for each variant individually. Details are here.
Your tool will help us a lot.
-
Thanks for taking the time to come up with a 'solution' to the problem
-
The bug has been fixed but it will take 24 hours for the caches to clear! Thanks again!
-
That is a weird bug, because our database is showing 18k+ for that term! I will investigate!
-
Thank you, Rand and Russ!
I found a weird result. "bag" in US is reported to have 0-10, while "bags" in US is 11.5k-30.3k.
Considering KW planner and Google trend, "bag" in US seems to be outlier.
-
You're working in the right direction, guys. It seems a little scammy on the part of search engines to show these search volumes, as if they prompt you to bid on those keywords because they have a high search volume. While they do show the same SERP for this keyword, I believe a page optimized with "Buy a car in Orlando" will have a better score and a lower bid than a page optimized for "Buys a cars in Orlandos".
-
Thanks!
-
You're welcome for our efforts. I am really excited as we roll out this volume fix. I think it is going to offer a real improvement for SEOs everywhere.
-
Thanks Russ and Rand! I've used many tools for getting to the bottom of estimated volume and have high hopes for your evolving KW Explorer. It's definitely needed.
-
This has a HUGE HISTORICAL impact.
-
Thanks for touching upon this topic which is seriously impacting all marketers who rely on google data. Even the core definition of "Search Volume" is now NOT correct. It's THAT bad as pointed out here http://bit.ly/29MvvvK. Thanks for your efforts.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Important news on Fresh Web Explorer
Hello Moz family, After much deliberation, Moz has arrived at the difficult decision to shift away from brand & keyword mentions monitoring as a standalone tool in Fresh Web Explorer. Though we believe in the value of this service for our users, we’ll no longer be supporting it in order to provide more innovative features that better suit the modern needs of SEOs. We will, however, continue to support existing features within the Pro platform that accomplish similar tasks, like link building, content planning, and mentions monitoring. We understand you might have a few questions and have compiled a list of common FAQs below. Within the answers, you will find a number of other resources to help you out. When is Moz sunsetting Fresh Web Explorer?
Moz Bar | | jacob.butler
Moz will be sunsetting Fresh Web Explorer on November 10, 2020. At that time, you will no longer be able to access this feature within Moz Pro. This means that there will no longer be the ability in your settings to opt into email alerts from Fresh Web Explorer as the email alerts will discontinue after this date as well. I used Fresh Web Explorer for brand and keyword monitoring across the web. How can I do that now? Though it’s not a direct replacement, setting Brand Rules within a Campaign allow a user to monitor specific keywords in the Link Opportunities section, much like Fresh Web Explorer. By monitoring the links that mention your tracked keywords, but do not link to them, you can easily reach out for link opportunities. It can also help you plan content strategies by understanding what is being discussed within the content of those pages. You can read more about Opportunities within Links here. I used Fresh Web Explorer for link building opportunities. How can I do that now? With Moz’s vast link index you can uncover valuable link prospects, create a solid strategy, and track your progress. Within Link Opportunities in a Campaign, you can understand the domains mentioning your tracked keywords (set within your Brand Rules), but not linking back to you. Link Intersect provides an analysis on who's linking to competitors, but not you, and can help users plan strategic link outreach. You can also use Link Tracking Lists to help track progress for link opportunities you’ve identified through unlinked mentions of your brand or your competitors. Our Keyword Explorer tool also includes a Mentions module. Mentions show you pages that contain the searched keyword and gives you a snapshot of the Page Title and URL. Within Mentions, understand a link’s Mention Authority score, which predicts the authority of the page using a proprietary algorithm, along with a combination of Moz metrics that take both the authority and influence of the page into account. Need help getting started with all of these similar features? Read more here. I used Fresh Web Explorer for content planning. How can I use Moz for that now? There are many ways you can plan your content strategy using Moz. For a high-level understanding of how Moz Pro can help you plan content, take a look at our Beginner’s Guide to Content Strategy here. Otherwise, one way you can understand what is being mentioned within your tracked keywords is through Opportunities in Links within a campaign. This feature helps you get a sense for what type of content is published on the web around your tracked keywords, branded or non-branded. You can then plan similar content as what’s being mentioned. For more information on how to set up Brand Rules in a Campaign, click here. I used Fresh Web Explorer for the email alerts to notify other stakeholders on keyword & brand mentions. Can I do that another way within Moz Pro? We recommend using our Custom Reports to email your stakeholders of tracked mentions on the web. You can set Brand Rules within your Campaign settings for branded or non-branded keywords you’d like to monitor. You can read more on how to set up Custom Reports here. The Moz Pro feature alternatives you suggested aren’t really a direct replacement. Why? We recognize that our feature suggestions are not a direct replacement for brand and keyword mentions monitoring. And, we also understand that this functionality is desired by many of our users. We will be exploring opportunities to include some of the Fresh Web Explorer functionality within future versions of Moz Pro. In the meantime, if you're looking for a direct replacement, we suggest checking out BuzzSumo. Their Monitoring feature provides you a one-stop-shop for monitoring your brand, keywords, and trends across the web. Do I need to do anything before the sunset on November 10, 2020? No. Fresh Web Explorer will simply disappear from the Moz Pro interface. No action is needed on your end. Once retired, you will no longer see the page within Moz Pro and you will no longer receive alert emails that were previously set up from the settings tab within an account. Do you have any content you can point me to that outlines the other ways in which I can find my brand’s mentions in Moz Pro? We sure do! Check out this fantastic Help Hub article that outlines all of the ways you might track, monitor, and analyze your brand’s mentions in Moz Pro. The features here include Keyword Explorer’s Mentions, Brand Rules, Link Opportunities in Links within a Campaign, Link Tracking Lists, and Custom Reports. If you have any questions, I'm happy to discuss more in the Q&A thread. Or, feel free to send an email to help@moz.com with any questions. Best,
Jacob Butler4 -
Why can I see 404 pages in Google Analytics but nothing in the On-Demand Crawl?
Hello, I'm looking at some Google Analytics data for a website and can see a few 'Page not found's among the Page Titles, looking like these are 404 errors. To get a full list of what's 404-ing so I can get these redirected, the Moz on-demand crawl of the website has come back with no major errors and just a few metadata ones. Does anyone know any potential reasons why the audit has drawn a blank, and is there another way to get a comprehensive list of 404s, as I'm aware the Google Analytics data may not be covering all of them. Thanks very much Becky
Moz Bar | | becky.jenkins0 -
How to export to csv all top pages links in link explorer?
When i press export to csv it does not offer to choose how many rows but exports to file only 500 rows. But there is more than 3000. How to export all rows? Thanks
Moz Bar | | AndrisZigurs0 -
Keyword explorer tool - keywords by domain functionality
Hi, We have issues with the new searching for keywords by root domain functionality in keyword explorer. We are based in Australia and need rankings for google.com.au, which is what we have set up within our MOZ campaign ranking reports. The explorer tool rankings are way off and not accurate for our site's actual rankings. Also, many of the top, high volume keywords that we are tracking and for which we rank on first page for are not even appearing when I do an explorer search using our root domain. Is the root domain functionality within keyword explorer providing results based on google.com rather than google.com.au? Does this mean that if we are outside the US that we can't use the tool as it is not accurate? Cheers, Damien
Moz Bar | | Devenish0 -
Is there a tool that works like crawl test that allows more than 3000 pages?
I enjoy using crawl test inside moz but I need to find a way to crawl all the pages on a site. It would probably be in the neighborhood of 10,000 urls. Does anyone know of a free tool and if not is there a paid tool that will do this?
Moz Bar | | bradwayland0 -
Keyword research and suggestion tool?
Does Moz have a keyword research and suggestion tool? We find articles on google keyword planner, etc. but what about Moz?
Moz Bar | | Manifestation0 -
Google Penalty Tools?
Does Anybody heard about this tool? http://fruition.net/google-penalty-checker-tool/ ? Why MOZ cannot provide the same tool with all the inner and analytic data Rogers have? Did you see a penalty tool alike? For the recent two updates, I think that there is no real "penalty" per-se. Most of the sites I checked lost position on some keywords, and gained better position on other sets of keywords. If this statement is true - We need no tool at all 😉
Moz Bar | | Elchanan1