Introduction
Yahoo Finance (finance.yahoo.com) is a media property that is part of the Yahoo! network.
It provides financial news, data, and commentary including stock quotes, press releases, financial reports, and original content.
In this article you will read about the easiest way to scrape stock prices and price changes from Yahoo Finance with Page2API.
You will find code examples for Ruby, Python, PHP, NodeJS, cURL, and a No-Code solution that will import stock pricing data into Google Sheets.
Prerequisites
To start scraping stock prices, we will need the following things:
- A Page2API account
-
A stock name (or a list of stocks) that we are interested in.
In our case, the stocks that we are interested in are Alphabet Inc. (GOOG), Amazon.com, Inc. (AMZN), Apple Inc. (AAPL), and Tesla, Inc. (TSLA).
How to scrape Yahoo Finance Stock Prices
First what we need is to open the URL for a stock price, ex: Apple Inc. (AAPL).
https://finance.yahoo.com/quote/AAPL
We will use this URL as the first parameter we need to start the scraping process.
The page that you see must look like the following one:
If you inspect the page HTML, you will find out that it looks like this:
From the Yahoo Finance Stock Price page, we will scrape the following attributes: - Name
- Price
- Change
- Change Percent
Now, let's define the selectors for each attribute.
/* Parent: */ (We will use it to scrape multiple stocks at once)
[data-testid=quote-hdr]
/* Name */
[data-testid=quote-hdr] h1
/* Price */
[data-testid=quote-hdr] [data-field='regularMarketPrice']
/* Change */
[data-testid=quote-hdr] [data-field='regularMarketChange']
/* Change Percent */
[data-testid=quote-hdr] [data-field='regularMarketChangePercent']
Now it's time to build the request that will scrape the stock prices.
The following examples will show how to scrape the data for a single stock: Apple Inc. (AAPL)
{
"api_key": "YOUR_PAGE2API_KEY",
"url": "https://finance.yahoo.com/quote/AAPL",
"parse": {
"name": "[data-testid=quote-hdr] h1 >> text",
"price": "[data-testid=quote-hdr] [data-field=regularMarketPrice] >> text",
"change": "[data-testid=quote-hdr] [data-field=regularMarketChange] >> text",
"change_percent": "[data-testid=quote-hdr] [data-field=regularMarketChangePercent] >> text"
}
}
require 'rest_client'
require 'json'
api_url = 'https://www.page2api.com/api/v1/scrape'
payload = {
api_key: 'YOUR_PAGE2API_KEY',
url: 'https://finance.yahoo.com/quote/AAPL',
parse: {
name: '[data-testid=quote-hdr] h1 >> text',
price: '[data-testid=quote-hdr] [data-field=regularMarketPrice] >> text',
change: '[data-testid=quote-hdr] [data-field=regularMarketChange] >> text',
change_percent: '[data-testid=quote-hdr] [data-field=regularMarketChangePercent] >> text'
}
}
response = RestClient::Request.execute(
method: :post,
payload: payload.to_json,
url: api_url,
headers: { "Content-type" => "application/json" },
).body
result = JSON.parse(response)
puts(result)
{
"result": {
"name": "Apple Inc. (AAPL)",
"price": "149.64",
"change": "+5.86",
"change-percent": "(+4.08%)"
}
}
If we decide to go scrape multiple stocks at once, we will go with the batch scraping approach.
{
"api_key": "YOUR_PAGE2API_KEY",
"batch": {
"urls": [
"https://finance.yahoo.com/quote/TSLA",
"https://finance.yahoo.com/quote/AAPL",
"https://finance.yahoo.com/quote/GOOG",
"https://finance.yahoo.com/quote/AMZN"
],
"concurrency": 1,
"merge_results": true
},
"parse": {
"stocks": [
{
"_parent": "[data-testid=quote-hdr]",
"name": "h1 >> text",
"price": "[data-field=regularMarketPrice] >> text",
"change": "[data-field=regularMarketChange] >> text",
"change_percent": "[data-field=regularMarketChangePercent] >> text"
}
]
}
}
require 'rest_client'
require 'json'
api_url = 'https://www.page2api.com/api/v1/scrape'
payload = {
api_key: 'YOUR_PAGE2API_KEY',
batch: {
urls: [
"https://finance.yahoo.com/quote/TSLA",
"https://finance.yahoo.com/quote/AAPL",
"https://finance.yahoo.com/quote/GOOG",
"https://finance.yahoo.com/quote/AMZN"
],
concurrency: 1,
merge_results: true
},
parse: {
stocks: [
{
_parent: "[data-testid=quote-hdr]",
name: "h1 >> text",
price: "[data-field=regularMarketPrice] >> text",
change: "[data-field=regularMarketChange] >> text",
change_percent: "[data-field=regularMarketChangePercent] >> text"
}
]
}
}
response = RestClient::Request.execute(
method: :post,
payload: payload.to_json,
url: api_url,
headers: { "Content-type" => "application/json" },
).body
result = JSON.parse(response)
puts(result)
{
"result": {
"stocks": [
{
"name": "Tesla, Inc. (TSLA)",
"price": "759.63",
"change": "+51.90",
"change-percent": "(+7.33%)"
},
{
"name": "Apple Inc. (AAPL)",
"price": "149.64",
"change": "+5.86",
"change-percent": "(+4.08%)"
},
{
"name": "Alphabet Inc. (GOOG)",
"price": "2,255.98",
"change": "+90.06",
"change-percent": "(+4.16%)"
},
{
"name": "Amazon.com, Inc. (AMZN)",
"price": "2,302.93",
"change": "+81.38",
"change-percent": "(+3.66%)"
}
]
}, ...
}
How to export Yahoo Finance Stock Prices to Google Sheets
To export our Stock Prices to a Google Spreadsheet, we will need to slightly modify our request to receive the data in CSV format instead of JSON.According to the documentation, we need to add the following parameters to our payload:
"raw": {
"key": "stocks", "format": "csv"
}
The URL with encoded payload will be:
Press 'Encode'
Note: If you are reading this article being logged in - you can copy the link above since it will already have your api_key in the encoded payload.
Press 'Encode'
The result must look like the following one:
Conclusion
All done!
Monitoring this kind of financial data can be quite valuable, simply because it will help you make better investment decisions.
With Page2API, the API that helps you scrape any data with ease, this process will become easy and fun.
