How to Scrape eBay Data: Products, Prices, and more

How to scrape eBay products

First what we need is to type 'fitbit 4 silicone strap' into the search input from eBay's search page change the view type to Gallery View.

This will change the browser URL to something similar to:

  
    https://www.ebay.com/sch/i.html?_nkw=fitbit+4+silicone+strap&_sacat=0&_dmd=2&rt=nc

The URL is the first parameter we need to perform the scraping.

The page that you see must look like the following one:

If you inspect the page HTML, you will find out that a single result is wrapped into a div that looks like the following:

The HTML for a single result element will look like this:

The last part is the pagination handling.

In our case, we must click on the Next (→) button while the list item's class will be active:

And stop our scraping request when the Next (→) button became disabled.

In our case, a new attribute (aria-disabled="true") is assigned to the Next (→) button:

Now it's time to prepare the request that will scrape all products that the search page returned.

  
    export API_KEY=YOUR_PAGE2API_KEY
  

  
    curl -XPOST -H "Content-type: application/json" -d '{
      "api_key": "'"$API_KEY"'",
      "url": "https://www.ebay.com/sch/i.html?_nkw=fitbit+4+silicone+strap&_sacat=0&_dmd=2&rt=nc",
      "real_browser": true,
      "merge_loops": true,
      "scenario": [
        {
          "loop" : [
            { "wait_for": ".pagination__next" },
            { "execute": "parse" },
            { "execute_js": "document.querySelector(\"a.pagination__next\").click()" }
          ],
          "stop_condition": "document.querySelector(\"a.pagination__next\").ariaDisabled == \"true\""
        }
      ],
      "parse": {
        "items": [
          {
            "_parent":"ul.srp-grid li.s-item",
            "title":"h3.s-item__title >> text",
            "link":"a.s-item__link >> href",
            "price":".s-item__price >> text",
            "shipping":".s-item__shipping >> text"
          }
        ]
      }
    }' 'https://www.page2api.com/api/v1/scrape' | python -mjson.tool
  

  
    {
      "result": {
        "items": [
          {
            "title": "For Fitbit Charge 3/4 Replacement Silicone Wristband Straps Sports Watch Band",
            "link": "https://www.ebay.com/itm/174485572199?hash=item28a0268a67:g:1QMAAOSwLA9fjTwQ",
            "price": "$3.89",
            "shipping": "Free shipping"
          },
          {
            "title": "Unisex Silicone Sports Bracelet Strap Fashion Band for Fitbit Inspire/Inspire HR",
            "link": "https://www.ebay.com/itm/324593039446?hash=item4b9340b856:g:5jsAAOSwRLJgh62j",
            "price": "$3.49",
            "shipping": "Free shipping"
          },
          ...
        ]
      },
      ...
    }
  

  
    https://www.ebay.com/itm/114503614713

  
    export API_KEY=YOUR_PAGE2API_KEY
  

  
    curl -XPOST -H "Content-type: application/json" -d '{
      "api_key": "'"$API_KEY"'",
      "url": "https://www.ebay.com/itm/114503614713",
      "parse": {
        "name": "h1 >> text",
        "price": "span[itemprop=price] >> text",
        "condition": "div[itemprop=itemCondition] >> text",
        "categories": [
          "#vi-VR-brumb-lnkLst span >> text"
        ],
        "amount_sold": ".soldwithfeedback a >> text",
        "seller_name": "#RightSummaryPanel span.mbg-nw >> text",
        "item_location": "span[itemprop=availableAtOrFrom] >> text",
        "feedback_score": "#RightSummaryPanel span.mbg-l a >> text",
        "approximate_price": "#convbidPrice >> text"
      }
    }' 'https://www.page2api.com/api/v1/scrape' | python -mjson.tool
  

  
    {
      "result": {
        "name": "Details about Fitbit Charge 2 3 4 Bracelet Stainless Steel Spare Band Nylon Milanese Sport",
        "price": "GBP 5.66",
        "condition": "New with tags",
        "categories": [
          "Jewelry & Watches",
          "Watches, Parts & Accessories",
          "Watch Accessories",
          "Wristwatch Bands"
        ],
        "amount_sold": "",
        "seller_name": "fro-shop",
        "item_location": "Mülheim, Germany",
        "feedback_score": "28166",
        "approximate_price": "US $7.76(including shipping)"
      },
      ...
    }