How To Bulk Index URLs To Google

Now this time we will learn How to Bulk Index URLs to the Search engine/ Google console for Indexing or Crawling purposes to help your Website SEO techniques.

You may see the need for the automation of this movement due to multiple reasons. For example, if the article is updated or article enhancement, etc. You usually try to index URLs to Google Search Engine using manual paths for your WordPress website or any other website.

Bulk Indexing will be a great SEO technique you can assume overall.

We shall cover the below factors in today’s post,

  •   Manual indexing and crawling problems
  •   Automatic indexing and crawling
  •   Submit multiple URLs to Google Search Engine
  •   Requirements for Automation – Request Permission
  •   Script for automation( Python,node.js, JAVA code, etc.)

Problems with Manual URL Indexing and Crawling

It is always hard to perform indexing manually like using the below search console window.

  • To submit URLs, you must be log-in every time.
  • Only authorized users like Admin only can index prosses after logging in.
  • It takes much time, and if you have multiple URLs to send, you will have to submit them one by one manually.
  • Crawling or indexing by Google can take some time ( wait may be more than days or weeks)
  • It’s possible that you overlooked URLs to submit for index (including modified or newly added posts)

Automation for Url Indexing or crawling for Google

Using an automation process, all of the above-mentioned challenges can be fixed. We’ll create a script to do authentication and permission in this paragraph.

Once approved, the script can be edited with new Links or existing modified links to submit or resubmit as posts are updated or resubmitted.

With this method, you can submit multiple URLs to google as per the quota limitation.

The quota limit is 20000 URLs as default.

multiple Index URLs to Google Search Engine

We shall cover the below set of steps required to be executed,

  • Create a Google Search console service account
  • Site Ownership- Add your service account as a website owner
  • with Written script and using an access token
  • Submit URL Programmatically

A standard manual step involves below,

  • Login to your Google Search Console account
  • Submit each URL individually one by one
  • and Wait for indexing to be effective

Before we begin to use the Indexing API, there is something that must be set up,

Authorize Requests to Indexing API

Application submissions must be permitted by an authenticated user who has access to that information.

Below we will cover high-level steps for performing the Authorization.

Create a Project

Before you can send submissions to the Indexing API, you need to inform Google about the customer and activate credentials to the API.

You do this by using the Google Console to create a new project or exiting project, which is a named group of settings and API access data, and register your application.

To get begun operating Indexing API, visit the setup tool, which instructs you through creating a new project in the Google API Console, enabling the API, and making credentials.

Create a service account

  • Now you have to go  Service accounts and Select your project else create a new project as explained above.
  • Click add the Create Service Account option and enter the name like the project name for easy to identify, and give a short description. You can work with the default service account ID, or choose a different, memorable one. When you are done then click Create and continue button.
Google Indexing API Submit URL using a script or batch file Create service account - Help Of Ai
Google Indexing API -Submit URL using a script or batch file Create service account
  • The Service account permissions (optional) area that follows is not required. Click Continue.
  • On the Grant users credentials to this service account. You can add a specific role or keep the role default owner.
  • Once done you have to see your service account create option as below,
  • In the main window panel, click on Action or: symbol. Please select the Manage keys option.

then select the format for your private key: JSON is recommended.

private key JSON - Help Of Ai
  • Click on Create. A new public/private key pair will be generated and downloaded automatically to your machine;

Add your service account as a website owner

For site ownership, please complete the below steps,

Prove as the Owner of the website 

The verification method of verifying that you own the property needs to be done by you.

Please note that verified owners will have access to sensitive Search data for a website.

Grant owner status to your service account 

The next stage is to create your Service account (created in the above step) as a site owner.

This step is an important step toward fetching actual access to Indexing API.

  • Please Open the Webmaster Console
  • Click on the verified website.
  • go to setting option
  • In the Verified owner list option, click Add an owner.
  • Provide your service account email as the delegated owner (before we created the service account email). You can locate your service account email address in two places:
    • client_email – This is an area in the JSON private key in the JSON service account file which we downloaded in one of the above stages.

An email has the below pattern,

<your-service-account>@<your-project-name>.google.com.iam.gserviceaccount.com

For example 

[email protected]
index URLs to Google Search Engine,Bulk Index URLs to Google SEO, index URL to Google Search Engine

Final Step- Bulk Submit URLs Programmatically

That’s all, you are now all ready to set up the automation. With needed requirements to start gathering the Google Indexing API.

As a next step, please use the scripts of Python (we are comfortable with Python) to call the Google Indexing API.

Please see the below link for the final stages where we will use a Python script to send the URL for Google indexing programmatically.

Please Download the Zip file to get the script for the indexing or crawling sense using automation processes (Now this time we have only one script).

The next step is to open your terminal and run these commands..

Windows user

1. pip install pip
2. pip install oauth2client httplib2
3. pip install pandas
Windows user - Help Of Ai
Windows user commands

Mac user

Code 1: pip3 install oauth2client httplib2
Code 2: pip3 install pandas

close your current terminal and go to the Folder that you have downloaded before, then unZip that file (IndexAPI). run command python indexing.py

Screenshot 2022 08 10 235117 - Help Of Ai
Screenshot 2022 08 10 234932 - Help Of Ai

all done, this script will do automation process for your URL indexing or crawling to search engine.

Screenshot 2022 08 10 235752 - Help Of Ai

Do you have any statements or ideas or any helpful suggestions to share?

Please write your comments below.

Picture of Hoa
Hoa

Leave a Comment