How to Conduct Bulk Domain Analysis with SE Ranking’s API

Analyzing thousands of domains for SEO insights can be tiresome. It often requires significant time and resources, but you can use API technology to make the process of conducting bulk domain checks more accessible and efficient. This makes it easier to analyze competitive niches and find better guest blogging opportunities. In this article, we will explore how SE Ranking’s API empowers users to perform comprehensive domain analysis at scale. You’ll discover how this translates to valuable insights that can greatly streamline your workflow and drive better results for your SEO projects. We will also look at the different ways you can integrate this API into your SEO routine.

Let’s jump right in!

TL;DR

An overview of APIs: what are they and how do they work?

The SE Ranking team has created several APIs. They were designed to simplify your work, both with large-scale domain data analytics and other SEO tasks. Now you can retrieve raw data without having to manually log into the SE Ranking platform. This is ideal for companies that manage tons of data and accounts (i.e., SEO agencies).

This API is available with a Business subscription only.

To analyze domains in bulk with SE Ranking, use the following two APIs: Competitive Research and Backlink Checker. 

The Competitor Research API presents a convenient format for accessing domain statistics from both organic and paid search results, including traffic, keywords, and other metrics. Meanwhile, the Backlink Checker API provides data that you can use to conduct backlink profile analyses. Both of these APIs combined offer nearly all the numeric metrics available on the platform versions of these tools.

Here are some of the key metrics available.

Competitive Research API:

• Total domain traffic within the specified location
• Number of keywords a domain ranks for in the specified location
• Number of newly acquired keywords
• Number of keywords that have dropped from the SERPs
• Number of keywords with consistent ranking positions
• Number of keywords with improved ranking positions
• Number of keywords with decreased ranking positions
• Total number of keywords ranked in positions 1-5, 6-10, and 11-20
• And more

Backlink Checker API:

• Domain Trust score
• Total number of backlinks for the URL
• Total number of referring domains for the URL
• Total number of dofollow/nofollow backlinks
• And more

This isn’t the complete list. You can find all parameters in our Backlink Checker API and Competitor Research API documents.

How to optimize your workflow with our API 

You don’t need to know how to code to fully utilize our APIs. Simply run the provided Python code and access the results through Google Sheets. 

Let’s begin by taking a closer look at the codes you will be working with.

Available codes

You can conduct a domain analysis using one of the following two scripts.

1. (Competitive Research + Backlink Checker API) 

You will need to copy and paste the code below into Google Colab. But please note that you must add your API key to the api_key parameter. This can be found within the API section of SE Ranking’s Settings. 

Competitive Research + Backlink Checker API code:

import requests
import gspread
from google.colab import auth, drive
from oauth2client.client import GoogleCredentials
import pandas as pd
from gspread_dataframe import set_with_dataframe
from google.auth import default
auth.authenticate_user()
creds, _ = default()


def read_table(table_name:str) -> pd.DataFrame:
gc = gspread.authorize(creds)
worksheet = gc.open(table_name).sheet1
rows = worksheet.get_all_values()
frame = pd.DataFrame.from_records(rows)
frame.columns = frame.iloc(0)
frame = frame.drop(index=(0))
frame = frame.fillna('')
return frame


def save_table(table_name:str, dataframe:pd.DataFrame) -> None:
  gc = gspread.authorize(creds)
  worksheet = gc.open(table_name).sheet1
  set_with_dataframe(worksheet, dataframe)


def request_to_api(research:pd.DataFrame) -> list:
   result = list()
   for index, row in research.iterrows():
       domain = row('domain')
       research_result = get_research_api_result(domain)
       backlinks_result = get_backlinks_api_result(domain) if (row('backlinks') == '') else row('backlinks')
       result.append((domain, research_result, backlinks_result))
   return result


def get_api_headers() -> str:
   api_key = 'ADD YOUR API KEY'
   return dict(Authorization=api_key)


def get_research_api_result(domain:str) -> str:
   params = dict(domain=domain)
   api_url="https://api4.seranking.com/research/uk/overview/?"
   result = requests.get(api_url, params=params, headers=get_api_headers())
   return result.json()


def get_backlinks_api_result(domain:str) -> str:
   report = create_backlink_checker_report(domain)
   report_id = str(report.get('report_id'))
   api_url="https://api4.seranking.com/backlink-reports/" + report_id + '/overview'
   result = requests.get(api_url, headers=get_api_headers())
   delete_backlink_checker_report(report_id)
   return result.json()


def create_backlink_checker_report(domain:str) -> str:
   params = dict(mode="domain", target=domain)
   api_url="https://api4.seranking.com/backlink-reports"
   result = requests.post(api_url, json=params, headers=get_api_headers())
   return result.json()


def delete_backlink_checker_report(report_id:str) -> None:
   api_url="https://api4.seranking.com/backlink-reports/" + report_id
   requests.delete(api_url, headers=get_api_headers())


def main():
   table_name="competitive-research-API"
   table = read_table(table_name)
   result_table_requests_to_api = request_to_api(table)
   result_table_to_dataframe = pd.DataFrame(result_table_requests_to_api, columns=('domain', 'result', 'backlinks'))
   save_table(table_name, result_table_to_dataframe)
main()

2. (Competitive Research API) 

Competitive Research code:

Don’t forget to input your API key into the api_key parameter.

import requests
import gspread
from google.colab import auth, drive
from oauth2client.client import GoogleCredentials
import pandas as pd
from gspread_dataframe import set_with_dataframe
from google.auth import default
auth.authenticate_user()
creds, _ = default()


def read_table(table_name:str) -> pd.DataFrame:
gc = gspread.authorize(creds)
worksheet = gc.open(table_name).sheet1
rows = worksheet.get_all_values()
frame = pd.DataFrame.from_records(rows)
frame.columns = frame.iloc(0)
frame = frame.drop(index=(0))
frame = frame.fillna('')
return frame


def save_table(table_name:str, dataframe:pd.DataFrame) -> None:
  gc = gspread.authorize(creds)
  worksheet = gc.open(table_name).sheet1
  set_with_dataframe(worksheet, dataframe)


def request_to_api(research:pd.DataFrame) -> list:
   result = list()
   for index, row in research.iterrows():
       domain = row('domain')
       research_result = get_research_api_result(domain)
       result.append((domain, research_result))
   return result


def get_api_headers() -> str:
   api_key = 'ADD YOUR API KEY'
   return dict(Authorization=api_key)


def get_research_api_result(domain:str) -> str:
  params = dict(domain=domain)
  api_url="https://api4.seranking.com/research/uk/overview/?"
  result = requests.get(api_url, params=params, headers=get_api_headers())
  return result.json()


def main():
  table_name="competitive-research-API"
  table = read_table(table_name)
  result_table_requests_to_api = request_to_api(table)
  result_table_to_dataframe = pd.DataFrame(result_table_requests_to_api, columns=('domain', 'result'))
  save_table(table_name, result_table_to_dataframe)
main()

Here are a few key parameters to pay attention to:

• This is how your API key appears in the code:

• In api_url, you can change the location from which to retrieve traffic/keyword data. 

• These are the columns in your Google Sheets file.

Now, let’s move on to our step-by-step guide to utilizing these codes.

How to use it

Step 1. Open Google Colab, create a new notebook and paste one of the two codes provided in the previous section.

Step 2. Open the Colab Notebooks folder in your Google Drive and create a Google Sheet file.

Note: Ensure that the file name is an exact match with the table_name in the code, as shown in the example below.

Step 3. Assign names to the columns based on the script being used. If using the Competitive Research API script exclusively, columns A and B should be called domain and result, respectfully. For the Competitive Research and Backlink Checker API scripts, in addition to the domain and result columns, you need to add a third column called backlinks. 

See the screenshot below showing the second version with three columns. 

If necessary, you can rename the columns and update the code accordingly.

Then, enter your domains in the first column without http:// and www.

Step 4: Go to Colab, click the button to run the code, and grant access if required.

Step 5: After the script finishes running, check back into the Google Sheets file at a later time to get the results. Here’s what the table will look like:

Step 6. To isolate the metric you specified, extract it from the JSON data. Just input the =REGEXEXTRACT function or come up with your own solution. Below is the formula for accessing the traffic_sum parameter, which represents the total organic traffic in a specified location:

=REGEXEXTRACT(B2,”traffic_sum’:\s((0-9)+)”)

• B2 is the column containing your JSON data (i.e., your result column). If you need data from another column, you can change it in the formula.
• traffic_sum is the metric of interest. If you want to retrieve other data, you can change this parameter in the formula. 
• \s((0-9)+) is the regular expression that searches for sequences where the whitespace character is immediately followed by one or more digits. You don’t need to change it. 

Step 7. Once you have entered the formula into the first cell, drag it down to copy the formula to all cells. Be sure to obtain data from each domain on the list.

Here is a list of key metrics that you can use in formulas to obtain important domain data:

• keywords_count — Total number of keywords a domain ranks for in the specified location.
• top1_5 — Total number of keywords that rank in the top 1-5.
• keywords_new_count — Total number of new keywords.
• domain_inlink_rank — Domain Trust score.
• total_domains — Total number of referring domains for a URL.
• total_backlinks — Total number of backlinks for a URL.

You can find each of these metrics in JSON format in our Backlink Checker API (for the backlinks column) and Competitor Research API (for the result column) documents.

Exploring API use cases

Wondering how can this data benefit you? Let’s explore some areas in SEO where you can benefit from API.

• Competitive and niche analysis

You can assess the strength of your competitors (or domains in your niche) based on factors such as organic traffic, number of keywords, backlinks, and domain trust. This helps you gauge the overall competitiveness of the niche and where you stand compared to these websites.

To expand your domain list and find more websites from within your niche, you can get SERP results for the keywords in your niche. Just use SE Ranking’s SERP Competitors feature under the My Competitors tab to get this information. 

Here’s how:

1. Select the desired keyword group from the dropdown menu (ensure that you have previously added all necessary keywords to a single group within the project).
2. Choose how many results from the SERP you need (100/50/30/20/10).
3. Select the Domain mode.

Then, export data. Select the All keywords from the group option to obtain a list of domains that rank in the top 10-100 for the keywords you want to examine.

After you have chosen the websites you want to examine, add them to the Google Sheets file and conduct a domain SEO analysis. It will compare the organic traffic, number of keywords, backlinks, referring domains, and domain trust scores of these sites. This can provide valuable insights into your niche’s competitive landscape.

To conduct an in-depth competitive analysis, use specialized tools like SE Ranking’s Competitive Research and Backlink Checker.

• Guest blogging and digital PR

You can also streamline your link building efforts by pinpointing the most relevant domains to contact for potential collaboration opportunities. Conducting a quick initial research study with API helps you evaluate domain quality before committing to investing in further resources. 

For instance, you can assess their organic traffic and Domain Trust scores to pinpoint potential sources for deeper analysis and further collaboration outreach. While this approach may not offer insights into site relevance and other important qualitative metrics, it is a good starting point for your campaign. 

Wrapping up

SE Ranking’s API functionality is an ideal solution for businesses managing multiple large-scale projects at once. It smoothens the process of analyzing huge datasets and automating SEO processes.

We hope that the API codes and guide we provided help simplify your bulk domain analysis efforts. We designed our APIs for everyone, even those without coding skills.