How to Use Google Indexing API to Submit URLs in Bulk using Python?

This guide is a non-programmer’s guide to using the Google Developers Indexing API to update or remove pages from the Google index instantly using Python. The objective of this guide is to use a python script to submit URLs in bulk to Google for crawling to fix some of your page indexing issues related to indexable URLs being crawled but currently not indexed. Follow the step-by-step guide illustrated with images to use Google Indexing API with Python.

Step-by-Step Guide to Using Google Indexing API through Python

Step 1: Install Python

Head to https://www.python.org/ to download and install the latest Python version. For the purpose of this guide, I would be installing the Windows version since I use a PC. But the process is similar for Mac. After the executable is downloaded, follow the self-explanatory installation to complete the installation.

Step 2: Setting up the Indexing API key in your Google Developer Console

Head to Google Developer Console. Google Indexing API is free. So do not worry about adding a billing account if you are just using the Indexing API.

• Create a New Project. Give it a name that is relevant and easy to remember. Since I am going to be using this for sorting indexing issues for Flannels, I’ll name the project ‘**** Indexing API’. Click on the Create Button.

• Head to the Newly Created Project (*** Indexing API project). After the new project is created, keep it as the selected project as shown in the screenshot below and follow the next steps.

• Setup APIs & Services Credentials. For this, you need to set up the API credentials by visiting the Credentials link under the APIs & Services menu using the left sidebar. Once you are on the Credentials window, click on ‘Manage Service Accounts’ and create a new Service Account.

• Create a Service Account. On the Create service account screen, you can follow the steps shown on the screen to set up your new service account. Please name the account with a memorable name that is relatable to your project. I am just going by the name ‘**** Indexing Service Account’ since this project is meant for ***. Once you add the name and click on create and continue, you will need to select a role to grant your service account access to the project. Please select the ‘Owner’ role and click the done button.

• Once you are done with the above step, you will be taken to the ‘Service accounts for the project screen as shown in the screenshot below. You will notice an email address (gservicesaccount.com email address) associated with your project and service account. Make a note of this email address as you will be using this email address in Step 3 of this guide. So, my advice is to save the email id onto a local notepad file.

• Create a Key for your Service Account. Click on the three dots under the actions column and select the ‘manage keys’ option. You will be taken to the ‘Keys’ screen where you can add a key by clicking on the ‘Add Key’ button. Keep the JSON option selected and click on the ‘Create’ button. A small JSON file will be downloaded. This is an authentication file which will verify your API access. A window displaying ‘Private key saved to your computer’ will appear which can be safely closed.

• Enable API. The next step is to enable the API. To do this, click on the ‘Library’ link under the API & Services menu section. Search for ‘indexing API’ and select the first Indexing API option from the two options shown. Click the ‘enable’ button on the next screen.

This completes all the steps of ‘Setting up the Indexing API key in your Google Developer Console’.

Step 3: Grant Owner Access to GServicesAccount Email in your Google Developer Console

In this step, you will grant owner-level access to the gservicesaccount.com email address that was created in the previous step. If you did not save the email address in the previous step, you can view the email address by selecting the Credentials link on the left sidebar. Once you have the email address copied, visit your Google Search Console account and select the website property that you want to use the indexing API. I have used it for Flannels as shown in the screenshot. You will need to be an owner of the property yourself in order to grant new ownership access to the gserviceaccount.com email.

Step 4: Final Step – Running the Python Script

So far, in the previous steps, you completed the set-up of the Indexing API key in your Google Developer Console, installed Python and granted owner access to the API key in your search console. Now we are in the final step where we will use the Google indexing API with python script.

• Install Some Libraries. Go to your Windows command prompt (search for CMD if you are using Windows or terminal if you are using a MAC). We will need to install some libraries before being able to use the indexing API. Run the following two commands on your command prompt. The httplib2 and the pandas’ library will download and install in your system.

For Windows Users

Command 1: pip install oauth2client httplib2

Command 2: pip install pandas

For Mac Users

Command 1: pip3 install oauth2client httplib2

Command 2: pip3 install pandas

• Create a folder to include 3 files – Python Script, URLs.csv and the JSON file that was previously downloaded. The JSON file that was downloaded in Step 2 will need to be moved into this folder. Download the urls.csv which will contain the list of URLs you want to submit to Google and the indexing.py python code using this link and move them all to one folder like the screenshot below. Full credit to the programmer for sharing the code here.
• Update the Python Code and the CSV File. The only change you need is to make a small change in line 8 of the indexing.py python code. Change the name of the JSON file on this line JSON_KEY_FILE = "YOUR_JSON_FILE.json" with the name of your JSON file. You can edit the file using software like Textpad or Notepad++. You can use Brackets on Mac. Next, you will need to add the URLs in column A of the urls.csv file. Do not remove the first row containing URL and Date values. You can add a maximum of 1000 URLs at a time.

• Execute the Script. Open the command prompt (on your PC) or terminal (if using a Mac) and navigate to the folder containing the 3 files. You can use the change directory CD command as shown in the screenshot below. Once you are in the right folder, you can execute the script using the below command.

For Windows Users

Command: python indexing.py

For Mac Users

Command: python3 indexing.py

Once you enter the command and press the enter button, you will see Google URL update notifications for all the URLs that were entered in your urls.csv one by one along with timestamps.