How to Scrape eBay Data: Products, Prices, and more

Introduction

eBay is an online shopping site that's best known for its auctions and consumer-to-consumer sales.

eBay scraping can be very useful if you intend to perform tasks such as:

analyzing products
predicting market trends
price monitoring

Compared to Amazon, scraping eBay can be a little more challenging, but the product page doesn't require a real browser, which leads to faster scraping speed.

To scrape eBay products, we will use Page2API - a powerful and delightful API that makes web scraping easy and fun.

In this article, we will learn how to:

Scrape eBay products
Scrape eBay product data

Prerequisites

To start scraping eBay, you will need the following things:

A Page2API account
The free trial offers a credit that covers up to 1000 web pages to scrape.

A product or a category of products that we are about to scrape.
In our case, we will search for 'fitbit 4 silicone strap'
and then scrape the product page for a random product.

How to scrape eBay products

First what we need is to type 'fitbit 4 silicone strap' into the search input from eBay's search page change the view type to Gallery View.

This will change the browser URL to something similar to:

  
    https://www.ebay.com/sch/i.html?_nkw=fitbit+4+silicone+strap&_sacat=0&_dmd=2&rt=nc

The URL is the first parameter we need to perform the scraping.

The page that you see must look like the following one:

If you inspect the page HTML, you will find out that a single result is wrapped into a div that looks like the following:

The HTML for a single result element will look like this:

The last part is the pagination handling.

In our case, we must click on the Next (→) button while the list item's class will be active:

And stop our scraping request when the Next (→) button became disabled.

In our case, a new attribute (aria-disabled="true") is assigned to the Next (→) button:

Now it's time to prepare the request that will scrape all products that the search page returned.

Setting the api_key as an environment variable

  
    export API_KEY=YOUR_PAGE2API_KEY

Running the scraping request with cURL

  
    curl -XPOST -H "Content-type: application/json" -d '{
      "api_key": "'"$API_KEY"'",
      "url": "https://www.ebay.com/sch/i.html?_nkw=fitbit+4+silicone+strap&_sacat=0&_dmd=2&rt=nc",
      "real_browser": true,
      "merge_loops": true,
      "scenario": [
        {
          "loop" : [
            { "wait_for": ".pagination__next" },
            { "execute": "parse" },
            { "execute_js": "document.querySelector(\"a.pagination__next\").click()" }
          ],
          "stop_condition": "document.querySelector(\"a.pagination__next\").ariaDisabled == \"true\""
        }
      ],
      "parse": {
        "items": [
          {
            "_parent":"ul.srp-grid li.s-item",
            "title":"h3.s-item__title >> text",
            "link":"a.s-item__link >> href",
            "price":".s-item__price >> text",
            "shipping":".s-item__shipping >> text"
          }
        ]
      }
    }' 'https://www.page2api.com/api/v1/scrape' | python -mjson.tool

The result

  
    {
      "result": {
        "items": [
          {
            "title": "For Fitbit Charge 3/4 Replacement Silicone Wristband Straps Sports Watch Band",
            "link": "https://www.ebay.com/itm/174485572199?hash=item28a0268a67:g:1QMAAOSwLA9fjTwQ",
            "price": "$3.89",
            "shipping": "Free shipping"
          },
          {
            "title": "Unisex Silicone Sports Bracelet Strap Fashion Band for Fitbit Inspire/Inspire HR",
            "link": "https://www.ebay.com/itm/324593039446?hash=item4b9340b856:g:5jsAAOSwRLJgh62j",
            "price": "$3.49",
            "shipping": "Free shipping"
          },
          ...
        ]
      },
      ...
    }

How to scrape eBay product data

From the products list page, we need to click on any product.

The browser URL will look like this:

  
    https://www.ebay.com/itm/114503614713

The URL is the first parameter we need to perform the product data scraping.

Let's prepare the request that will scrape the needed information from the page.

Setting the api_key as an environment variable (if needed)

  
    export API_KEY=YOUR_PAGE2API_KEY