How to Scrape Glassdoor Reviews

Introduction

Glassdoor.com is an American website where current and former employees anonymously review companies.

In this article, you will read about the easiest way to scrape Glassdoor reviews with Page2API.

You will find code examples for Ruby, Python, PHP, NodeJS, cURL, and a No-Code solution that will import Glassdoor reviews into Google Sheets.

DISCLAIMER: we highly recommend you to scrape Glassdoor only for personal use.
For example: let's say you are looking for a new job and you want to quickly analyze the reviews for the company that you are interested in.

Prerequisites

To start scraping Glassdoor reviews, we will need the following things:

A Page2API account
A company name that we are interested in.
In our case, the company that we are interested in is... Glassdoor.
(Which also has reviews on its website)

How to scrape Glassdoor Reviews

First what we need is to open glassdoor.com and type Glassdoor reviews into the search input.

This will change the browser URL to something similar to:

  
    https://www.glassdoor.com/Reviews/Glassdoor-Reviews-E100431.htm

We will use this URL as the first parameter we need to start the scraping process.

The page that you see must look like the following one:

If you inspect the page HTML, you will find out that a single review looks like the following:

From the Glassdoor Reviews page, we will scrape the following attributes from each review:

Title
Author Info
Rating
Pros
Cons
Helpful

Now, let's define the selectors for each attribute.

  
    /* Parent: */
    div.gdReview

    /* Title */
    a.reviewLink

    /* Author Info */
    .authorInfo

    /* Rating */
    span.ratingNumber

    /* Pros */
    span[data-test=pros]

    /* Cons */
    span[data-test=cons]

    /* Helpful */
    div.common__EiReviewDetailsStyle__socialHelpfulcontainer

Let's handle the pagination.
There are two approaches that can help us scrape all the needed pages:

1. We can iterate through the pages by clicking on the Next page button
2. We can scrape the pages using the batch scraping feature

Let's take a look at the first approach.

To go to the next page, we must click on the next page button if it's present on the page:

  
    document.querySelector(".nextButton").click()

The scraping will continue while the next page button is present on the page, and stop if it disappears.
The stop condition for the scraper will be the following javascript snippet:

  
    document.querySelector(".nextButton") === null

Now it's time to build the request that will scrape Glassdoor reviews.

The following examples will show how to scrape 5 pages of reviews from Glassdoor.com

If we decide to go with the next button approach, our payload will look like:

  
    {
      "api_key": "YOUR_PAGE2API_KEY",
      "url": "https://www.glassdoor.com/Reviews/Glassdoor-Reviews-E100431.htm",
      "real_browser": true,
      "merge_loops": true,
      "premium_proxy": "us",
      "scenario": [
        {
          "loop": [
            { "wait_for": "div.gdReview" },
            { "execute": "parse" },
            { "execute_js": "document.querySelector(\".nextButton\").click()" }
          ],
          "iterations": 5,
          "stop_condition": "document.querySelector('.nextButton') === null"
        }
      ],
      "parse": {
        "reviews": [
          {
            "_parent": "div.gdReview",
            "title": "a.reviewLink >> text",
            "author_info": ".authorInfo >> text",
            "rating": "span.ratingNumber >> text",
            "pros": "span[data-test=pros] >> text",
            "cons": "span[data-test=cons] >> text",
            "helpful": "div.common__EiReviewDetailsStyle__socialHelpfulcontainer >> text"
          }
        ]
      }
    }

Code examples (next button approach)

      
    require 'rest_client'
    require 'json'

    api_url = 'https://www.page2api.com/api/v1/scrape'
    payload = {
      api_key: 'YOUR_PAGE2API_KEY',
      url: 'https://www.glassdoor.com/Reviews/Glassdoor-Reviews-E100431.htm',
      real_browser: true,
      merge_loops: true,
      premium_proxy: 'us',
      scenario: [
        {
          loop: [
            { wait_for: 'div.gdReview' },
            { execute: 'parse' },
            { execute_js: 'document.querySelector(".nextButton").click()' }
          ],
          iterations: 5,
          stop_condition: 'document.querySelector(".nextButton") === null'
        }
      ],
      parse: {
        reviews: [
          {
            _parent: 'div.gdReview',
            title: 'a.reviewLink >> text',
            author_info: '.authorInfo >> text',
            rating: 'span.ratingNumber >> text',
            pros: 'span[data-test=pros] >> text',
            cons: 'span[data-test=cons] >> text',
            helpful: 'div.common__EiReviewDetailsStyle__socialHelpfulcontainer >> text'
          }
        ]
      }
    }

    response = RestClient::Request.execute(
      method: :post,
      payload: payload.to_json,
      url: api_url,
      headers: { "Content-type" => "application/json" },
    ).body

    result = JSON.parse(response)

    puts(result)

      
    import requests
    import json

    api_url = 'https://www.page2api.com/api/v1/scrape'
    payload = {
      "api_key": "YOUR_PAGE2API_KEY",
      "url": "https://www.glassdoor.com/Reviews/Glassdoor-Reviews-E100431.htm",
      "real_browser": True,
      "merge_loops": True,
      "premium_proxy": "us",
      "scenario": [
        {
          "loop": [
            { "wait_for": "div.gdReview" },
            { "execute": "parse" },
            { "execute_js": "document.querySelector(\".nextButton\").click()" }
          ],
          "iterations": 5,
          "stop_condition": "document.querySelector(\".nextButton\") === null"
        }
      ],
      "parse": {
        "reviews": [
          {
            "_parent": "div.gdReview",
            "title": "a.reviewLink >> text",
            "author_info": ".authorInfo >> text",
            "rating": "span.ratingNumber >> text",
            "pros": "span[data-test=pros] >> text",
            "cons": "span[data-test=cons] >> text",
            "helpful": "div.common__EiReviewDetailsStyle__socialHelpfulcontainer >> text"
          }
        ]
      }
    }

    headers = {'Content-type': 'application/json', 'Accept': 'text/plain'}
    response = requests.post(api_url, data=json.dumps(payload), headers=headers)
    result = json.loads(response.text)

    print(result)

      
    <?php

    $api_url = 'https://www.page2api.com/api/v1/scrape';
    $payload = [
      'api_key' => 'YOUR_PAGE2API_KEY',
      'url' => 'https://www.glassdoor.com/Reviews/Glassdoor-Reviews-E100431.htm',
      'real_browser' => true,
      'merge_loops' => true,
      'premium_proxy' => 'us',
      'scenario' => [
        0 => [
          'loop' => [
            0 => [ 'wait_for' => 'div.gdReview' ],
            1 => [ 'execute' => 'parse' ],
            2 => [ 'execute_js' => 'document.querySelector(\'.nextButton\').click()' ]
          ],
          'iterations' => 5,
          'stop_condition' => 'document.querySelector(\'.nextButton\') === null'
        ]
      ],
      'parse' => [
        'reviews' => [
          0 => [
            '_parent' => 'div.gdReview',
            'title' => 'a.reviewLink >> text',
            'author_info' => '.authorInfo >> text',
            'rating' => 'span.ratingNumber >> text',
            'pros' => 'span[data-test=pros] >> text',
            'cons' => 'span[data-test=cons] >> text',
            'helpful' => 'div.common__EiReviewDetailsStyle__socialHelpfulcontainer >> text'
          ]
        ]
      ]
    ];

    $postdata = json_encode($payload);
    $ch = curl_init($api_url);
    curl_setopt($ch,CURLOPT_POST, true);
    curl_setopt($ch,CURLOPT_POSTFIELDS, $postdata);
    curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type:application/json'));
    curl_setopt($ch,CURLOPT_RETURNTRANSFER, true);
    $result = curl_exec($ch);
    curl_close($ch);

    echo $result;

    ?>

      
    const axios = require('axios');
    const api_url = 'https://www.page2api.com/api/v1/scrape';
    const payload = {
      api_key: 'YOUR_PAGE2API_KEY',
      url: 'https://www.glassdoor.com/Reviews/Glassdoor-Reviews-E100431.htm',
      real_browser: true,
      merge_loops: true,
      premium_proxy: 'us',
      scenario: [
        {
          loop: [
            { wait_for: 'div.gdReview' },
            { execute: 'parse' },
            { execute_js: 'document.querySelector(".nextButton").click()' }
          ],
          iterations: 5,
          stop_condition: 'document.querySelector(".nextButton") === null'
        }
      ],
      parse: {
        reviews: [
          {
            _parent: 'div.gdReview',
            title: 'a.reviewLink >> text',
            author_info: '.authorInfo >> text',
            rating: 'span.ratingNumber >> text',
            pros: 'span[data-test=pros] >> text',
            cons: 'span[data-test=cons] >> text',
            helpful: 'div.common__EiReviewDetailsStyle__socialHelpfulcontainer >> text'
          }
        ]
      }
    };

    axios.post(api_url, payload)
      .then((res) => {
        console.log(JSON.stringify(res.data, null, "  "));
      }).catch((err) => {
         console.error(err);
      });

      
    curl -XPOST -H "Content-type: application/json" -d '{
      "api_key": "YOUR_PAGE2API_KEY",
      "url": "https://www.glassdoor.com/Reviews/Glassdoor-Reviews-E100431.htm",
      "real_browser": true,
      "merge_loops": true,
      "premium_proxy": "us",
      "scenario": [
        {
          "loop": [
            { "wait_for": "div.gdReview" },
            { "execute": "parse" },
            { "execute_js": "document.querySelector(\".nextButton\").click()" }
          ],
          "iterations": 5,
          "stop_condition": "document.querySelector(\".nextButton\") === null"
        }
      ],
      "parse": {
        "reviews": [
          {
            "_parent": "div.gdReview",
            "title": "a.reviewLink >> text",
            "author_info": ".authorInfo >> text",
            "rating": "span.ratingNumber >> text",
            "pros": "span[data-test=pros] >> text",
            "cons": "span[data-test=cons] >> text",
            "helpful": "div.common__EiReviewDetailsStyle__socialHelpfulcontainer >> text"
          }
        ]
      }
    }' 'https://www.page2api.com/api/v1/scrape' | python -mjson.tool

If we decide to go with the batch scraping approach, our payload will look like:

  
    {
      "api_key": "YOUR_PAGE2API_KEY",
      "real_browser": false,
      "premium_proxy": "us",
      "batch": {
        "urls": "https://www.glassdoor.com/Reviews/Glassdoor-Reviews-E100431_P[1, 5, 1].htm",
        "concurrency": 1,
        "merge_results": true
      },
      "parse": {
        "reviews": [
          {
            "_parent": "div.gdReview",
            "title": "a.reviewLink >> text",
            "author_info": ".authorInfo >> text",
            "rating": "span.ratingNumber >> text",
            "pros": "span[data-test=pros] >> text",
            "cons": "span[data-test=cons] >> text",
            "helpful": "div.common__EiReviewDetailsStyle__socialHelpfulcontainer >> text"
          }
        ]
      }
    }

Code examples (batch scraping approach)

      
    require 'rest_client'
    require 'json'

    api_url = 'https://www.page2api.com/api/v1/scrape'
    payload = {
      api_key: 'YOUR_PAGE2API_KEY',
      real_browser: false,
      premium_proxy: 'us',
      batch: {
        urls: "https://www.glassdoor.com/Reviews/Glassdoor-Reviews-E100431_P[1, 5, 1].htm",
        concurrency: 1,
        merge_results: true
      },
      parse: {
        reviews: [
          {
            _parent: 'div.gdReview',
            title: 'a.reviewLink >> text',
            author_info: '.authorInfo >> text',
            rating: 'span.ratingNumber >> text',
            pros: 'span[data-test=pros] >> text',
            cons: 'span[data-test=cons] >> text',
            helpful: 'div.common__EiReviewDetailsStyle__socialHelpfulcontainer >> text'
          }
        ]
      }
    }

    response = RestClient::Request.execute(
      method: :post,
      payload: payload.to_json,
      url: api_url,
      headers: { "Content-type" => "application/json" },
    ).body

    result = JSON.parse(response)

    puts(result)

      
    import requests
    import json

    api_url = 'https://www.page2api.com/api/v1/scrape'
    payload = {
      "api_key": "YOUR_PAGE2API_KEY",
      "real_browser": False,
      "premium_proxy": "us",
      "batch": {
        "urls": "https://www.glassdoor.com/Reviews/Glassdoor-Reviews-E100431_P[1, 5, 1].htm",
        "concurrency": 1,
        "merge_results": True
      },
      "parse": {
        "reviews": [
          {
            "_parent": "div.gdReview",
            "title": "a.reviewLink >> text",
            "author_info": ".authorInfo >> text",
            "rating": "span.ratingNumber >> text",
            "pros": "span[data-test=pros] >> text",
            "cons": "span[data-test=cons] >> text",
            "helpful": "div.common__EiReviewDetailsStyle__socialHelpfulcontainer >> text"
          }
        ]
      }
    }

    headers = {'Content-type': 'application/json', 'Accept': 'text/plain'}
    response = requests.post(api_url, data=json.dumps(payload), headers=headers)
    result = json.loads(response.text)

    print(result)

      
    <?php

    $api_url = 'https://www.page2api.com/api/v1/scrape';
    $payload = [
      'api_key' => 'YOUR_PAGE2API_KEY',
      'url' => 'https://www.glassdoor.com/Reviews/Glassdoor-Reviews-E100431.htm',
      'real_browser' => false,
      'merge_loops' => true,
      'premium_proxy' => 'us',
      'batch' => [
        'urls' => 'https://www.glassdoor.com/Reviews/Glassdoor-Reviews-E100431_P[1, 5, 1].htm',
        'concurrency' => 1,
        'merge_results' => true
      ],
      'parse' => [
        'reviews' => [
          0 => [
            '_parent' => 'div.gdReview',
            'title' => 'a.reviewLink >> text',
            'author_info' => '.authorInfo >> text',
            'rating' => 'span.ratingNumber >> text',
            'pros' => 'span[data-test=pros] >> text',
            'cons' => 'span[data-test=cons] >> text',
            'helpful' => 'div.common__EiReviewDetailsStyle__socialHelpfulcontainer >> text'
          ]
        ]
      ]
    ];

    $postdata = json_encode($payload);
    $ch = curl_init($api_url);
    curl_setopt($ch,CURLOPT_POST, true);
    curl_setopt($ch,CURLOPT_POSTFIELDS, $postdata);
    curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type:application/json'));
    curl_setopt($ch,CURLOPT_RETURNTRANSFER, true);
    $result = curl_exec($ch);
    curl_close($ch);

    echo $result;

    ?>

      
    const axios = require('axios');
    const api_url = 'https://www.page2api.com/api/v1/scrape';
    const payload = {
      api_key: 'YOUR_PAGE2API_KEY',
      real_browser: false,
      premium_proxy: 'us',
      batch: {
        urls: 'https://www.glassdoor.com/Reviews/Glassdoor-Reviews-E100431_P[1, 5, 1].htm',
        concurrency: 1,
        merge_results: true
      },
      parse: {
        reviews: [
          {
            _parent: 'div.gdReview',
            title: 'a.reviewLink >> text',
            author_info: '.authorInfo >> text',
            rating: 'span.ratingNumber >> text',
            pros: 'span[data-test=pros] >> text',
            cons: 'span[data-test=cons] >> text',
            helpful: 'div.common__EiReviewDetailsStyle__socialHelpfulcontainer >> text'
          }
        ]
      }
    };

    axios.post(api_url, payload)
      .then((res) => {
        console.log(JSON.stringify(res.data, null, "  "));
      }).catch((err) => {
         console.error(err);
      });

      
    curl -XPOST -H "Content-type: application/json" -d '{
      "api_key": "YOUR_PAGE2API_KEY",
      "real_browser": false,
      "premium_proxy": "us",
      "batch": {
        "urls": "https://www.glassdoor.com/Reviews/Glassdoor-Reviews-E100431_P[1, 5, 1].htm",
        "concurrency": 1,
        "merge_results": true
      },
      "parse": {
        "reviews": [
          {
            "_parent": "div.gdReview",
            "title": "a.reviewLink >> text",
            "author_info": ".authorInfo >> text",
            "rating": "span.ratingNumber >> text",
            "pros": "span[data-test=pros] >> text",
            "cons": "span[data-test=cons] >> text",
            "helpful": "div.common__EiReviewDetailsStyle__socialHelpfulcontainer >> text"
          }
        ]
      }
    }' 'https://www.page2api.com/api/v1/scrape' | python -mjson.tool

The result

  
    {
      "result": {
        "reviews": [
          {
            "title": "Glassdoor Walks the Walk",
            "author_info": "Jan 7, 2022 - Senior Manager",
            "rating": "5.0",
            "pros": "Glassdoor creates a positive environment for employees to learn and grow. ...",
            "cons": "At any organization, there is always room for improvement. ...",
            "helpful": "1 person found this review helpful"
          },
          {
            "title": "Great Company To Work For",
            "author_info": "Jan 5, 2022 - Customer Success Manager",
            "rating": "4.0",
            "pros": "I absolutely love working at Glassdoor. ...",
            "cons": "While we do have more of an extensive career growth plan, ...",
            "helpful": "2 people found this review helpful"
          }, ...
        ]
      }, ...
    }

How to export Glassdoor reviews to Google Sheets

In order to be able to export our Glassdoor reviews to a Google Spreadsheet we will need to slightly modify our request to receive the data in CSV format instead of JSON.

According to the documentation, we need to add the following parameters to our payload:

  
    "raw": {
      "key": "reviews", "format": "csv"
    }

Now our payload will look like:

{ "api_key": "YOUR_PAGE2API_KEY", "real_browser": false, "premium_proxy": "us", "batch": { "urls": [ "https://www.glassdoor.com/Reviews/Glassdoor-Reviews-E100431_P1.htm", "https://www.glassdoor.com/Reviews/Glassdoor-Reviews-E100431_P2.htm", "https://www.glassdoor.com/Reviews/Glassdoor-Reviews-E100431_P3.htm", "https://www.glassdoor.com/Reviews/Glassdoor-Reviews-E100431_P4.htm", "https://www.glassdoor.com/Reviews/Glassdoor-Reviews-E100431_P5.htm" ], "concurrency": 1, "merge_results": true }, "raw": { "key": "reviews", "format": "csv" }, "parse": { "reviews": [ { "_parent": "div.gdReview", "title": "a.reviewLink >> text", "author_info": ".authorInfo >> text", "rating": "span.ratingNumber >> text", "pros": "span[data-test=pros] >> text", "cons": "span[data-test=cons] >> text", "helpful": "div.common__EiReviewDetailsStyle__socialHelpfulcontainer >> text" } ] } }

Please note that the batch URLs are defined explicitly to make it simpler to edit the payload.

Now, edit the payload above if needed, and press Encode →

Encode

The URL with encoded payload will be:

  Press 'Encode'

Note: If you are reading this article being logged in - you can copy the link above since it will already have your api_key in the encoded payload.

The final part is adding the IMPORTDATA function, and we are ready to import our Glassdoor reviews into a Google Spreadsheet.

  Press 'Encode'

The result must look like the following one:

Glassdoor reviews import to Google Sheets

Conclusion