A year ago, in April of 2015, I pitched a project internally at Moz to design and launch a keyword research tool, one of the few areas of SEO we've never comprehensively tried to serve. The pitch took effort and cajoling (the actual, internal pitch deck is available here), but eventually received approval, with one big challenge... We had to do it with a team already dedicated to the maintenance and development of our rankings collections and research tools. This project wouldn't get additional staffing — we had to find a way to build it with only the spare bandwidth of this crew.
Sure, we didn't have the biggest team, or the ability to work on the project free from our other obligations, but we had grit. We had passion. We wanted to prove ourselves to our fellow Mozzers and to our customers. We had pride. And we desperately wanted to build something that wasn't just "good enough," but was truly great. Today, I think we've done that.
Meet our new keyword research tool, Keyword Explorer:
If you want to skip hearing about it and just try it out, head on over. You can run 2 free searches/day without even logging in, another 5 with a free community account, and if you're a Pro subscriber, you've already got access. For those who want to learn more, read on!
The 5 big, unique features of Keyword Explorer
Keyword Explorer (which we've taken to calling "KWE" for short) has lots of unique features, metrics, and functionality, but the biggest ones are pretty obvious and, we believe, highly useful:
KWE takes you all the way through the keyword research process — from discovering keyword ideas to getting metrics to building a list, filtering the keywords on it, and prioritizing which ones to target based on the numbers that matter.
KWE features metrics essential to the SEO process — two you're familiar with — Volume and Difficulty — and three that are less familiar: Opportunity, Importance, and Potential. Opportunity estimates the relative CTR of the organic web results on a SERP. Importance is a metric you can modify to indicate a keyword that's more or less critical to your campaign/project. And Potential is a combination of all the metrics built to help you prioritize a keyword list.
Our volume score is the first volume estimation metric we know of that goes beyond what AdWords reports. We do that using Russ Jones' volume bucket methodology and adding in anonymized clickstream data from ~1 million real searchers in the US. From there, Russ has built a model that predicts the search volume range a keyword is likely to have with ~95% accuracy.
Keyword suggestions inside KWE come from almost all the sources we saw SEOs accessing manually in their research processes — Keyword Planner data, Google Suggest, Related Searches, other keywords that the ranking pages also ranked for, topic-modeling ideas, and keywords found from our clickstream data. All of these are available in KWE's suggestions.
Import and export functionality are strongly supported. If you've already got a list of keywords and just want KWE's metrics, you can easily upload that to us and we'll fetch them for you. If you like the KWE process and metrics, but have more you want to do in Excel, we support easy, powerful, fast exports. KWE is built with power users in mind, so go ahead and take advantage of the tool's functionality however works best with your processes.
These five are only some of the time-saving, value-adding features in the tool, but they are, I think, enough to make it worthwhile to give Keyword Explorer a serious look.
A visual walkthrough
As an experiment, I've created a visual, slide-by-slide walkthrough of the tool. If you'd rather *see* vs. read the details, this format might be for you (Slideshare's embedding is having issues today, but you can find the presentation on their site).
And, for those of you who prefer video, we made a short, 2 minute demo of the tool in that format, too:
Of course, there's a ton of nuance and complexity in a product like this, and given Moz's dedication to transparency, you can find all of that detail in the more thorough explanation below.
Keyword Explorer's metrics
KWE's metrics are among the biggest data-driven advances we've made here at Moz, and a ton of credit for that goes to Dr. Pete Meyers and Mr. Russ Jones. Together, these two have crafted something extraordinary — unique metrics that we've always needed for SEO-based keyword research, but never had before. Those include:
Keyword volume ranges
Nearly every keyword research tool available uses a single source for volume data: Google AdWords' Keyword Planner. We all know from studying it that the number AdWords provides is considerably off from reality, and last year, Moz's Russ Jones was able to quantify those discrepancies in his blog post: Keyword Planner's Dirty Secrets.
Since we know that Google's numbers don't actually have precision, but do indicate a bucket, we realized we could create ranges for volume and be significantly more accurate, more of the time. But, that's not all... We also have access to anonymized clickstream data here at Moz, purchased through a third-party (we do NOT collect or use any of our own user data via, for example, the MozBar), that we were able to employ in our new volume ranges.
Using sampling, trend data, and the number of searchers and searches for a given keyword from the clickstream, combined with AdWords' volume data, we produced a volume range that, in our research, showed ~95% accuracy with the true impression counts Google AdWords would report for a keyword whose ad showed during a full month.
We're pretty excited about this model and the data it produces, but we know it's not perfect yet. As our clickstream data grows, and our algorithm for volume improves, you should see more and more accurate ranges in the tool for a growing number of keywords. Today, we have volume data on ~500mm (half a billion) English-language search queries. But, you'll still see plenty of "no data" volume scores in the tool as we can access considerably more terms and phrases for keyword suggestions (more on suggestion sources below).
NOTE: KWE uses volume data modeled on the quantity of searches in the US for a given term/phrase (global English is usually 1.5-3X those numbers). Thus, while the tool can search any Google domain in any country, the volume numbers will always be for US-volume. In the future, we hope to add volume data for other geos as well.
An upgraded, more accurate Keyword Difficulty score
The old Keyword Difficulty tool was one of Moz's most popular (it's still around for another month or so, but will be retired soon in favor of Keyword Explorer). But, we knew it had a lot of flaws in its scoring system. For Keyword Explorer, we invested a lot of energy in upgrading the model. Dr. Pete, Dr. Matt Peters, myself, and Russ had 50+ reply email threads back and forth analyzing graphs, suggesting tweaks, and tuning the new score. Eventually, we came up with a Keyword Difficulty metric that:
Has far more variation than the old model — you'll see way more scores in the 20s and 30s as well as the 80s and 90s than the prior model, which put almost every keyword between 50–80.
Accounts for pages that haven't yet been assigned a PA score by using the DA of the domain.
Employs a smarter, CTR-curve model to show when weaker pages are ranking higher and a page/site may not need as much link equity to rank.
Adjusts for a few domains (like Blogspot and Wordpress) where DA is extremely high, but PA is often low and the inherited domain authority shouldn't pass on as much weight to difficulty.
Concentrates on however many results appear on page 1, rather than the top 20 results.
This new scoring model matches better with my own intuition, and I think you'll find it vastly more useful than the old model.
As you can see from one of my lists above (for Haiku Deck, whose board I joined this year), the difficulty ranges are considerably higher than in the past, and more representative of how relatively hard it would be to rank in the organic results for each of the queries.
A true Click-Through Rate Opportunity score
When you look at Google's results, it's pretty clear that some keywords are worthy of pursuit in the organic web results, and some are not. To date, no keyword research tool we know of has attempted to accurately quantify that, but it's a huge part of determining the right terms and phrases to target.
Once we had access to clickstream data, we realized we could accurately estimate the percent of clicks on a given search result based on the SERP features that appeared. For example, a classic, "ten-blue-links" style search result had 100% of click traffic going to organic results. Put a block of 4 AdWords ads above it, though, and that dropped by ~15%. Add a knowledge graph to the right-hand side and another ~10% of clicks are drawn away.
It would be crazy to treat the prioritization of keywords with loads of SERP features and little CTR on the organic results the same as a keyword with few SERP features and tons of organic CTR, so we created a metric that accurately estimates Click-Through-Rate (CTR), called "Opportunity."
The search above for "Keanu" has an instant answer, knowledge graph, news results, and images (further down). Hence, its Opportunity Score is a measly 37/100, which means our model estimates ~37% of clicks go to the organic results.
But, this search, for "best free powerpoint software" is one of those rare times Google is showing nothing but the classic 10 blue links. Hence, its Opportunity Score is 100/100.
If you're prioritizing keywords to target, you need this data. Choosing keywords without it is like throwing darts with a blindfold on — someone's gonna get hurt.
Importance scores you can modify
We asked a lot of SEOs about their keyword research process early in the design phases of Keyword Explorer and discovered pretty fast that almost everyone does the same thing. We put keyword suggestions from various sources into Excel, get metrics for all of them, and then assign some type of numeric representation to each keyword based on our intuition about how important it is to this particular campaign, or how well it will convert, or how much we know our client/boss/team desperately wants to rank for it.
That self-created score was then used to help weight the final decision for prioritizing which terms and phrases to target first. It makes sense. You have knowledge about keywords both subjective and objective that should influence the process. But it needs to do so in a consistent, numeric fashion that flows with the weighting of prioritization.
Hence, we've created a toggle-able "Importance" score in Keyword Explorer:
After you add keywords to a list, you'll see the Importance score is, by default, set to 3/10. We chose this number to make it easy to increase a keyword's importance by 3X and easy to bring it down to 1/3rd. As you modify the importance value, overall Keyword Potential (below) will change, and you can re-sort your list based on the inputs you've given.
For example, in my list above, I set "free slideshow software" to 2/10, because I know it won't convert particularly well (the word "free" often does not). But, I also know that churches and religious organizations love Haiku Deck and find it hugely valuable, so I've bumped up the importance of "worship presentation software" to 9/10.
Keyword Potential
In order to prioritize keywords, you need a metric that combines all the others — volume, difficulty, opportunity, and importance — with a consistent, sensible algorithm that lets the best keywords rise to the top. In Keyword Explorer, that metric is "Potential."
Sorting by Potential shows me keywords that have lots of search volume, relatively low difficulty, relatively high CTR opportunity, and uses my custom importance score to push the best keywords to the top. When you build a list in Keyword Explorer, this metric is invaluable for sorting the wheat from the chaff and identifying the terms and phrases with the most promise.
Keyword research & the list building process
Keyword Explorer is built around the idea that, starting from a single keyword search, you can identify suggestions that match your campaign's goals and include them in your list until you've got a robust, comprehensive set of queries to target.
List building is easy — just select the keywords you like from the suggestions page and use the list selector in the top right corner (it scrolls down as you do) to add your chosen keywords to a list, or create a new list:
Once you've added keywords to a list, you can go to the lists page to see and compare your sets of keywords:
Each individual list will show you the distribution of metrics and data about the keywords in it via these helpful graphs:
The graphs show distributions of each metric, as well as a chart of SERP features to help illustrate which types of results are most common in the SERPs for the keywords on your list:
For example, you can see in my Rock & Grunge band keywords, there's a lot of news results, videos, tweets, and a few star reviews, but no maps/local results, shopping ads, or sitelinks, which makes sense. Keyword Explorer is using country-level, non-personalized, non-geo-biased results, and so some SERPs won't match perfectly to what you see in your local/logged-in results. In the future, we hope to enable even more granular location-based searches in the tool.
The lists themselves have a huge amount of flexibility. You can sort by any column, add, move, or delete in bulk, filter based on any metric, and export to CSV.
If your list gets stale, and you need to update the metrics and SERP features, it's just a single click to re-gather all the data for every keyword on your list. I was particularly impressed with that feature; to me it's one of the biggest time-savers in the application.
Keyword Explorer's unique database of search terms & phrases
No keyword research tool would be complete without a massive database of search terms and phrases, and Keyword Explorer has just that. We started with a raw index of over 2 billion English keywords, then whittled that down to the ~500 million highest-quality ones (we collapsed lots of odd suggestions we found via iterative crawls of AdWords, autosuggest, related searches, Wikipedia titles, topic modeling extractions, SERPscape — via our acquisition last year — and more) into those we felt relatively confident had real volume).
Keyword Explorer's suggestions corpus features six unique filters to get back ideas. We wanted to include all the types of keyword sources that SEOs normally have to visit many different tools to get, all in one place, to save time and frustration. You can see those filters at the top of the suggestions page:
SERPs Analysis
The final feature of Keyword Explorer I'll cover here (there are lots of cool nooks and crannies I've left for you to find on your own) is the SERPs Analysis. We've broadened the ability of our SERP data to include all the features that often show up in Google's results, so you'll see a page much more representative of what's actually in the keyword SERP:
Holy smack! There's only 3 — yes, THREE — organic results on page one for the query "Disneyland." The rest is sitelinks, tweets, a knowledge graph, news listings, images — it's madness. But, it's also well-represented in our SERPs Analysis. And, as you can see, the Opportunity score of "7" effectively represents just how little room there is for organic CTR.
Over time, we'll be adding and supporting even more features on this page, and trying to grab more of the metrics that matter, too (for example, after Twitter pulled their tweet counts, we had to remove those from the product and are working on a way to get them back).
