Keyword Research

K

Keywords can be viewed as the individual paths your customers take from the search box to get to your product or service. Keyword Research, thus stands for the processes behind finding all the relevant keywords and being able to structure them in a manner that allows for anticipation of exactly what Search Queries your customers will use when searching for either your type of business or topics of interest.

Keyword Research is at the cornerstone of SEO, as organic search rankings ultimately revolve around keywords and are what enables the process of search and its’ optimisation to take place.

Why Is Keyword Research Important?

If you publish a page on a topic that a lot of people find interesting, that article puts you in a good position to receive a lot of traffic from search engines like Google. However because it is actually slightly more complicated, we need to address our efforts to Keyword Research.

Ahrefs Organic Traffic Study of 1 Billion Pages

You might be surprised, but according to a study conducted by Ahrefs, 90% of web pages on the Internet receive no organic search traffic from Google. So the question is how do you approach Keyword Research to be among the lucky 10%?

Ahrefs Organic Traffic Study of 1 billion pages

At its simplest, Keyword Research helps you ensure that there is a Search Demand for whatever you want to write about. At a higher level Keyword Research also ensures that your page has a high chance of ranking well in Search Engines given the competitiveness of the target keyword. This way you’ll be enjoying a consistent stream of highly targeted visitors to your page. At the upper-most level, Keyword Research also aims to ensure that the content you write will be in line with users’ Search Intent.

So how can you be a part of the minority that gets Organic Search Traffic from Search Engines and Google in particular? The Search Engine Ranking Algorithms count at least in the hundreds of factors that may prevent you from ranking or uplifting your pages in Google search results. However, there are a number of factors that all those non-ranking pages shared, which tie in closely to Keyword Research efforts:

1. Lack of Backlinks

It would be an understatement to say that backlinks help you rank higher in Google, but they remain one of Google’s top 3 ranking factors. So there is little surprise that there is a direct correlation between the number and quality of backlinks and Organic Rankings. Let’s take a look at the trend between the page’s monthly organic search traffic and its backlinks from unique websites (Referring Domains):

Ahrefs Organic Traffic Study of 1 Billion Pages

According to the same study conducted by Ahrefs, there appears to be a direct correlation between the Number of Referring Domains leading to a page and the Organic Search Traffic.

referring domains search traffic

This chart poses the question of whether it’s possible to rank for target keywords, having no links to the page and the answer is that it really depends on one thing. Firstly, it really helps if the page that is without backlinks is part of a website with high authority, meaning that it has backlinks to other pages on the same website. The stronger the overall authority of the website, the stronger the likelihood of other pages without backlinks to rank. This also doesn’t come without ‘ifs”. In this case, when you have a website that is authoritative in a niche, it’s really important to use the power of internal linking to pass down link-juice to pages without backlinks.

In simple terms, this relates to Keyword Research because the lower the authority of your domain the more out of your way you will have to go to target keywords you’ll actually be able to rank for and the more in-depth your Keyword Research should go. In other words, if you lack backlinks you will have to avoid targeting Head Terms in favour of Longtail Search Queries.

2. Lack of Traffic Potential

Since we’ve explored deep some exceptions of pages that rank without backlinks, what about the pages that have a lot of backlinks but do not rank? The answer is simple – those pages are not targeting a topic that people are searching for. In other words, if few people are searching for whatever you detail on your page, you will get limited Organic Search Traffic for that page.

Ahrefs Organic Traffic Study of 1 Billion Pages

The same study conducted by Ahrefs clearly supports the previous graphs, but also shows that there are always exceptions. Despite the fact that Landing Pages with more backlinks are more likely to rank, there are pages with many backlinks that do not rank at all. This leads to the conclusion that these pages aren’t targeting popular-enough search queries.

referring-domains-search-traffic-negative

In other words, when setting up your Keyword Research Strategy, you will learn to find keywords that have Traffic Potential. You will be able to ensure that the topics you will be writing about are and will continue to be searched by your potential customers.

3. Non-Matching Search Intent

Search Engines aim to return the most relevant search results for a query. That’s why the top organic results for “best niche product” are likely to be blog posts with recommendations and not product pages. In other words, search engines know that searchers are in research mode, not buying mode and thus present a set of results that are in line with their Search Intent. However, if you have pages with lots of backlinks but no Organic Traffic and they already target a keyword with traffic potential—another quick SEO win is to re-optimise them for Search Intent.

Determine Search Intent in SEMRush

Search Intent is one of the most difficult aspects to get right in Keyword Research, but it doesn’t have to be. The long-used method of finding Search Intent is simply going to the search results and capturing the Intent behind each and every listing on the first page of SERPs. This however has changed with SEMRush which can now help you determine Search Intent for any keyword automatically.

Head Terms and Longtail Keywords

At its basics, keywords can be of two kinds: Head Terms and Longtail Keywords. The main difference between the two is the length of the search query, where Head Terms are shorter than Longtail keywords which, respectively, are longer because they aim to address more complex or narrowed-down topics of interest.

Although not without exceptions, a meaningful difference between the two is that Head Terms are normally used to search for a service directly, which makes them appear either navigational or transactional when it comes down to Search IntentLongtail Keywords normally fall under the conversational or informational intent, one that seeks to start a conversation or seeks more in-depth information on a topic of interest, respectively.

The Search Demand Curve

According to Ahrefs, Longtail Keywords got their name from their position on the search demand curve. If we put together a list of all search queries that people have performed in a search engine over the course of a month and order them by their search volumes on a graph, it’ll look like this.

The Search Demand Curve by Ahrefs

Thus, at the left side of the curve, we have a tiny number of keywords with super high search volumes which fall under Head Terms, while the right side consists of billions of keywords with very low search volumes, referred to as Longtail keywords.

Keyword Search Metrics

As part of keyword research, SEO Managers use a set of primary metrics which can be called Keyword Metrics and which include the following:

  1. Keyword Search Volume
  2. Keyword Difficulty
  3. Traffic Potential
  4. Cost per Click (Paid Search)

Keyword Search Volume shows the amount of searches a keyword receives over a limited period of time. Keyword Difficulty specifies how difficult it will be to rank for a particular keyword. Estimated Inbound Organic Traffic anticipates the number of visitors a URL will bring to the website from one or a group of keywords that it ranks for. Lastly, cost per click (CPC) is associated with paid search and is useful in the context that it answers the question of how much are paid searchers willing to pay to rank for this keyword.

Ahrefs Keyword Metrics

These metrics are the primary metrics which are likely to be included in any SEO software. However, they were made on the basis of Ahrefs as the industry leader in Keyword Research Software.

1. Keyword Search Volumes

As previously stated, Keyword Search Volume shows the amount of searches a keyword receives over a limited period of time. SEO Managers will pay attention to keyword search volume averages when determining how popular a keyword is. This information is then used to guide their strategy on what keywords are most likely to garner interest and drive traffic.

It’s important to note that Keyword Search Volumes may vary depending on the industry. So the size of your site and your budget should factor into your research. If you’re running a large corporate website with a high budget, you may be capable of spending more on highly competitive keywords to maintain your competitive edge. However, if you’re new around the the block, you will be likely to choose a strategy that goes after the keywords with lower search volume because of a lower competition for them.

What SEO Managers are often trying to find is a combination of high Keyword Search Volume with a low Keyword Difficulty, thus plugging gaps in the market. Using overly popular terms may end up with your website ranking very low for these keywords, if it doesn’t have the authority, while using rarely searched terms means very few people may end up finding your content to begin with. So finding the right balance is paramount. 

It may also be important to note that Keyword Search Volumes are averages. This means that Keyword Search Volumes may be flat or seasonal as well as on a downward or upward trend. Evergreen keywords are searched for consistently, regardless of the period while Seasonal keywords are time-sensitive.  Similarly, you’ll probably want to avoid keywords that are on a downward trend and target those that are on an upward trend, respectively.

Google Trends

One tool used widely across the SEO community to determine the search trend for virtually any keyword is Google Trends, which happens to be free of charge. For instance “Keyword Research” seems to be on an Upward Trend for the timespan of the last 5 years.

Screenshot 2023 10 27 at 17.47.15

2. Keyword Difficulty

Keyword Difficulty is a metric by Ahrefs that comes in as a quantifiable source of truth for how difficult it would be to rank a web page for a particular keyword.

Ahrefs Keyword Difficulty Scale

How many referring domains you’ll need to rank in Top 10 search results for a keyword

Ahrefs Keyword Difficulty Scale

It may be worth noting that Ahrefs Keyword Difficulty Score is a rather simplistic measure of how hard it would be to rank for a particular keyword on purpose. There are other factors at stake, such as how indepth the landing pages might be on your target keywords that the tool doesn’t take into account.

In fact, most All-in-One SEO tools have a Keyword Difficulty score which is most of the time developed in-house using different sets of parameters. This is the reasons you may find wildly different Keyword Difficulty Scores among different tools. This is the reason it is always recommended checking the SERPs manually for your competition as in a good amount of cases Keyword Difficulty may not be the most trustworthy indicator of how hard it actually is to rank for a Search Query.

3. Traffic Potential

Lastly, one of the most difficult things to do as part of the keyword research process is estimating the actual inbound organic traffic potential to your website. This is, however, not easy to obtain for several reasons. Firstly, not all search results pages get the same amount of clicks.

Depending on SERP features, the amount of clicks given to organic search results varies significantly. For instance, conversational search queries that pose a question may be directly answered in SERPs by the search engine and thus would prevent the searchers from clicking on search results for further information.

Secondly, SERPs with transactional search queries, again, would direct a good amount of traffic to paid search results thus limiting the traffic potential of the SERP for organic search results.

Lastly, not only is it the amount of clicks that differs among SERPs, but calculating the traffic potential based on keywords would be simply wrong as it is a metric different from Search volume in that it measures the traffic directed to particular landing pages displayed in SERPs and those can be accessed through a multiple number of different keywords, not just one.

4. Cost per Click (CPC) and Paid Search

A rather underestimated value in SEO comes in the form of cost per click (CPC) estimates. These estimates prove to be indicative of the strengths of the transactional intent behind the search query.

In simple terms CPC shows how much advertisers are willing to pay to rank on page one for any given keyword, thus presenting value to the SEO function in that it helps to determine how well the keyword will ultimately convert. The higher the associated CPC values – the higher, it can be assumed, that the conversion rate for that keyword is.

The main assumption with cost per click (CPC) is that if someone is willing to pay for the keyword, it holds transactional value. However, it must be noted that this value may not be the same for everyone, it really depends on everyone’s individual case.

Keyword Search Intent

It’s difficult to stress out just how important search intent is for SEO and it is not an exaggeration to say that if you want to rank in organic SERPPs, understanding and creating content with search intent in mind can be crucial. The answer to what is search intent is the reason behind every search query.  Every keyword has a Search Intent which can be one or a combination of the following:

  • Navigational
  • Transactional
  • Conversational
  • Informational

As you already know, Head Phrases are normally used to search for a service directly, which makes them, most of the time, appear either navigational or transactional. Longtail Keywords normally fall under the conversational or informational intent, one that seeks to start a conversation or seeks more in-depth information on a topic of interest, respectively.

The easiest or in fact the only way of checking the search intent for a search query when doing keyword research is to to type it in a search engine and manually review the results. Although search intent is often evident from the search query itself, it is not always the case which is why it is important to check it manually.

Determine Search Intent in SEMRush

Search Intent is one of the most difficult aspects to get right in Keyword Research, but it doesn’t have to be. The long-used method of finding Search Intent is simply going to the search results and capturing the Intent behind each and every listing on the first page of SERPs. This however has changed with SEMRush which can now help you determine Search Intent for any keyword automatically.

Search intent should pretty much dictate the type of content you create. For keywords that have Navigational Search Intent, ensure you have your brand name visible in every page title. In the case of Transactional Search Intent, create a product landing page. When targeting Conversational Search Intent, make sure you have a related section on your website, along with internal links within your content that would assist users in fully solving a search query. When targeting Informational Search Intent, write a comprehensive article that fully solves the search query.

Keyword Discovery

Keyword research begins with thinking about how potential customers might be searching for your business or website. You can then employ keyword research tools to extend the list of your so-called “seed” keyword ideas to incorporate other “related” search terms.

The keyword discovery process begins with setting apart a limited list of keywords that will then be used for brainstorming. As an extension, you can also have a peek at your competitors’ keywords, those that already bring traffic. Even if you’re well familiar with your industry, looking for keyword ideas from your competition can go a long way in discovering new keyword ideas. Once you’ve established a list of keyword ideas your website as well as your competition already rank for, you may want to extend into those that neither you nor your competitors do.

Ahrefs Keyword Explorer

There are a number of keyword discovery tools, however, we’ll use Ahrefs’ Keyword Explorer as the leader among them based on the number of keywords provided along with the targeting options available. This is already enough to generate a certain amount of keyword ideas to work with.

Ahrefs Keywords Explorer

To extend this list you may also turn to industry forums; groups and Q&A sites, however, that’s for when you’ve exhausted the first option. It is also not out of the question to experiment with these and go back and forth as you learn.

Keyword Strategy

The last step in targeting keywords is grouping them according to semantics that can be meaningful for further prioritising them based on various criteria. This involves a strategy called Keyword Clustering. Keyword Clustering allows for the segmentation of a group of different keywords relevant to every web page on the website.

The Keyword Clustering is based on the Top 10 search results, regardless of the search engine or custom settings. Keyword Clustering works in the following way: when two or more keywords lead to the same page being shown under the Top 10 search results, these keywords are clustered together. As a result, the targeting is done for multiple keywords but through the same landing page. There are a number of automated SEO tools that perform keyword clustering.

Finally, when going through the keyword research process, it is important to keep in mind that your main task is spotting keywords with the highest return on investment and not simply those that seem easy to rank for. When it comes to Keyword Strategy, It is advisable to have both short and long-term ranking goals. This way ensuring you go after the most lucrative keywords from the very start ensuring that you are building sustainable content marketing practices early.

Keyword Mapping

Keyword Mapping fuels the practice of both Content Marketing (for new content) and Onpage Optimisation (for existing content). Both Content Marketing and Onpage Optimisation garner the efforts geared towards writing new content and optimising existing content for a set of particular keywords. Keyword Mapping surfaces hypotheses for Onpage Optimisation which are then applied on a per-landing-page basis.

In other words, Keyword Mapping involves attributing keywords to web pages based on their propensity to positively influence organic rankings. Thus, in a few words, Keyword Mapping stands for the practice of specifying which of the new or existing web pages should rank for what keywords.

This is, however, not limited to web pages only. Among other data points being mapped to the selected keywords are the Search Intent, as well as the Primary Keyword Metrics including Search Volumes, Keyword Difficulty, Organic Traffic, and CPC. Some high-level trends are the direct correlation, for instance, between the search volumes and the length of the search queries, or the correlation between search volumes and keyword difficulty.

Similarly, web pages may come with additional data points like the Onpage Elements themselves. Some of the possible data points for web pages you can include are the following Onpage Elements:

  • Page Title
  • Page Meta Description
  • H1
  • Body Content Word Count

These allow for analyses of the correlations between the Primary Search Metrics and the Optimisation of Onpage Elements. These are the sort of particularities that make up the Keyword Analysis and define the Keyword Strategy at large. For example, a low-established website with low rankings is likely to require longer page titles targeting long-tail search queries, as those that have lower search volumes and are likely to have lower keyword difficulty.

The Optimisation of Onpage Elements alone, however, isn’t nearly enough for a Keyword Strategy to be put in place. In order for the specified Web Pages to obtain rankings for their corresponding keywords, one must employ an Offsite Strategy. One, that will bring inbound links to the website from authoritative and trustworthy sources, serving as a testament that the website at hand is really going to deliver on the promise they set out with their Onpage Elements.

Keyword Strategy Measurability

Looking at the SEO measurability at large and keyword research dimensions and metrics in particular, in the context of the organic digital acquisition funnel the outcomes of keyword research can be clustered into the following dimensions:

Organic Visibility stands for the absolute reach of the website within SERPs related to all search queries (keywords) within each of the targeted search engines. The visibility may vary depending on SERP fluctuations as well as how deep within SERPs the users may dive into, but for practical reasons it is often limited to SERPs’ 1st to 10th pages.

Organic Rankings are the actual seats all landing pages occupy in the SERPs. Organic Rankings fluctuate over time depending at large, on search engine algorithm updates and timing nuances, but also on content and technical changes to websites as well as changes in user behaviour triggered by their evolving characteristics and preferences.

Organic Performance is actuated by the website’s propensity to sculpt a thorough and healthy organic visibility by establishing sustainable organic rankings within SERPs. In a commercial setting, organic performance is ultimately defined by the effectiveness of the website to attract the right users at scale by serving the right content at the right time. In simple terms, this can be described as Organic Traffic.