How to Create a System for Bulk URL Submission to Google Index and Checking Site Indexing

In this article, we’ll explain in detail how we created the system, the steps we followed, and provide working code examples.

Introduction

Every website owner faces the need to monitor page indexing and accelerate the addition of new URLs to Google. In a highly competitive environment, it’s crucial for new content or product pages to appear in search results as quickly as possible, rather than waiting weeks for a search bot to discover them.

Manually checking indexing and submitting links through Google Search Console quickly becomes a tedious and time-consuming task, especially if you run an online store with hundreds of products or a blog with regular posts.

We decided to automate this process using Google’s tools. For bulk URL submission to the index, we used the Google Indexing API, and for subsequent checks of page presence in search results — the Google Custom Search API. This approach allows:

  • simplifying bulk indexing of new pages;

  • instantly checking which pages are already in search and which are not;

  • monitoring the effectiveness of SEO efforts.



1. Creating a Google Project and Service Account

The first step — preparing an account for API usage:

  1. Create a project in Google Cloud Console

    1. Go to console.cloud.google.com and create a new project (e.g., example-index). - link . 

Creating a project in Google

Project name

2. Create a Service Account
In the IAM & Admin → Service Accounts section, create a new service account.

  • Assign a name (e.g., vitaminator-com),

  • Select the role Owner  

 

Configuration

The next window will appear, where you don’t need to specify anything; just click the Done button.

Click Done

Create and Download JSON Key
After creating the service account, generate a key in JSON format. It contains:

  • client_email — service account email,

  • private_key — private key,

  • other authentication data.

 

Creating a key

Creating a key

 

Creating JSON

Example key:

 {
  "type": "service_account",
  "project_id": "example-index",
  "private_key_id": "abc123",
  "private_key": "-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\n",
  "client_email": "example@example-index.iam.gserviceaccount.com",
  ...
}

 

2. Enable Google Indexing API
     In the APIs & Services → Library section, find Indexing API and enable it for the project. here

Indexing API and enabling it for the project

Node.js Script for Sending URLs to Google Index

We used Node.js and the googleapis library for authentication via JWT.

 

const fs = require('fs'); const { google } = require('googleapis'); const request = require('request'); const key = require('./service_account.json'); const jwtClient = new google.auth.JWT( key.client_email, null, key.private_key, ['https://www.googleapis.com/auth/indexing'] ); const batch = fs.readFileSync('urls.txt').toString().split('\n'); jwtClient.authorize((err, tokens) => { if (err) return console.error('Auth error:', err); const items = batch.map(url => ({ 'Content-Type': 'application/http', 'Content-ID': '', body: 'POST /v3/urlNotifications:publish HTTP/1.1\n' + 'Content-Type: application/json\n\n' + JSON.stringify({ url, type: 'URL_UPDATED' }) })); const options = { url: 'https://indexing.googleapis.com/batch', method: 'POST', headers: { 'Content-Type': 'multipart/mixed' }, auth: { bearer: tokens.access_token }, multipart: items }; request(options, (err, resp, body) => { if (err) return console.error('Error sending batch:', err); console.log('✅ Google Index API Response:\n', body); }); });

After running the script, each link is sent to Google Index. Example response:

{ "urlNotificationMetadata": { "url": "https://vitaminator.com.ua/product/bluebonnet-nutrition-120" } }

Checking Indexing with Custom Search API

To check if a URL is in the index:

Create a Custom Search Engine (CSE):

Create it here - here 

  • Enter the site example.com,

  • Restrict the search to this domain only.

Custom Search Engine

Custom Search Engine cx

 

Obtain API Key  (search engine identifier).

https://console.cloud.google.com/apis/credentials

get api

api number

You should end up with:


Api key = AIzaSyAk3234234234d-gZAk8wmJE4IGFo_w8
CX = e35e2de1234566c5

Script for checking indexing in Google: 


const fs = require('fs');
const axios = require('axios');
const urls = fs.readFileSync('urls.txt').toString().split('\n');
const apiKey = 'YOUR_API_KEY';
const cx = 'YOUR_CX';

(async () => {
  for (const url of urls) {
    try {
      const resp = await axios.get('https://www.googleapis.com/customsearch/v1', {
        params: { key: apiKey, cx, q: `site:${url}` }
      });

      const status = resp.data.items && resp.data.items.length > 0 ? '✅ Indexed' : '❌ Not indexed';
      console.log(`${url} — ${status}`);
      fs.appendFileSync('index-check-results.txt', `${url} — ${status}\n');
      
      // Pause for 1 second to avoid exceeding limits
      await new Promise(r => setTimeout(r, 1000));
    } catch (e) {
      console.error('Error checking:', e.response?.data || e.message);
    }
  }
})();

Limits and Quotas

  • Google Indexing API — 200 requests per day by default.

  • Custom Search API — free quota of 100 requests per day.

  • To avoid exceeding limits:

    • Use pauses between requests,

    • Split URLs into batches,

    • Request a quota increase through Google Cloud Console.

 

We created a fully automated process:

  • Sending URLs to Google Index,

  • Checking indexing for each page,

  • Saving results to a file,

  • Option to integrate with other systems (CMS, Telegram bot, Google Sheets).

This speeds up SEO processes, saves time, and makes indexing monitoring transparent.

After obtaining all APIs and Keys

We prepared a ready-to-use archive where you only need to insert the obtained keys.
 
Download link  - here

 

Instructions for Installing and Using the Indexing Check and URL Submission Script

Requirements

  1. Installed Node.js (version 18+).
    Download here: https://nodejs.org/.
    After installation, verify:

     
    node -v npm -v

  2. An account in Google Cloud Console.

    • Enabled Indexing API.

    • Created Service Account and downloaded JSON key.

    • Enabled Custom Search API and created a search engine (CSE) for your site.

Installation

  1. Unpack the archive to a convenient folder.

    For example:

     
    cd ~/Desktop/exemple

    2. Install dependencies:

     
    npm install

    🔑 Key Configuration

    1. Service Account Key (Indexing API)

    • File service_account.json — insert your JSON key from Google Service Account here.

    • Example content:

     
    { "type": "service_account", "project_id": "vitaminor-index", "private_key_id": "abc123", "private_key": "-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\n", "client_email": "your-service-account@project-id.iam.gserviceaccount.com" }

    ⚠️ This file must not be shared publicly; it is for your use only.

    Custom Search API Keys

    Open the file chekindex.js and replace the lines:

     
    const apiKey = "YOUR_API_KEY"; // Google API Key const cx = "YOUR_CX"; // Custom Search Engine ID

    with your values.

    • API Key: Obtained from Google Cloud Console → APIs & Services → Credentials.

    • CX: Obtained from your Custom Search Engine settings.

    📄 Files

    • urls.txt — list of URLs for checking and sending to the index.
      Each address on a new line.

    • index.js — script for sending links to Google Index API.

    • chekindex.js — script for checking indexing via Custom Search API.

    • index-check-results.txt — results of the checks are automatically saved here.

    Sending Links to Google Index

    Run the script from the folder, command: 
     
    node index.js
    Example response:✅ Google Index API Response: { "urlNotificationMetadata": { "url": "https://vitaminator.com.ua/product/example" } }

    2. Checking Link Indexing

     
    node chekindex.js
    Example result:

    https://vitaminator.com.ua/product/bluebonnet-nutrition-120 — ✅ Indexed

    https://vitaminator.com.ua/product/1000-tart-cherry-extract-solgar-90 — ❌ Not indexed

    All results are saved to the file index-check-results.txt

    Done!

    Now you have an automated tool for:

    • sending pages to Google Index,

    • checking their indexing,

    • saving results to a file.

    Result

    script execution result

    Thanks to the Brander team for inspiration and some materials that helped in writing this article.