Jsoup Stock Quote Scraping Yahoo Finance

Question

Using OracleSQL and Java

I have a "TickerSymbol Database" and a "Stockquote Database".

Selecting TickerSymbols from "GOOG","APPL","FB", and "AMZN" from the "TickerSymbolDatabase"
and circulating the ticker symbols at the end of YahooFinance URL. http://finance.yahoo.com/q?s= (TICKER)

Then finding the stock quote, and inserting the quote data into the "Stockquote Database".

I'm not exactly sure how to use the Jsoup selector, or how to circulate the ticker symbols at the end of the YahooFinance URL

Jonathan Hedley · Accepted Answer · 2013-11-18T06:24:05.077

1

Here's a simple example. Please check the TOS, and you might prefer Stanley's suggestion of fetching via CSV. I wanted to show how to fetch it in jsoup. Getting it into Oracle is a different question.

String[] codes = {"TSLA", "F", "TM"};
String baseUrl = "http://finance.yahoo.com/q?s=";
String ua = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/537.33 (KHTML, like Gecko) Chrome/27.0.1438.7 Safari/537.33";

for (String code : codes) {
    String url = baseUrl + code;
    Document doc  = Jsoup.connect(url).userAgent(ua).timeout(10*1000).get();
    String price = doc.select(".time_rtq_ticker").first().text();
    String name = doc.select(".title h2").first().text();

    System.out.println(String.format("%s [%s] is trading at %s", name, code, price));
}

This outputs:

Tesla Motors, Inc. (TSLA) [TSLA] is trading at 135.45
Ford Motor Co. (F) [F] is trading at 17.07
Toyota Motor Corporation (TM) [TM] is trading at 127.98

I like to use Try jsoup to test and debug URL responses and selectors queries.

edited Nov 18 '13 at 06:24

answered Nov 18 '13 at 06:18

Jonathan Hedley

10,442
3
36
47

Thanks that really helps, I will keep on investigating – user3003451 Nov 18 '13 at 07:30
Great. BTW if this answered your original question, don't forget to hit accept; that helps others know it's done. – Jonathan Hedley Nov 18 '13 at 14:37
retrieving 1 single value usually isn't that hard. But have you tried accessing the history data ? - that data is loaded with javascript so JSoup can't really help. There is a CSV download as well, but the a-href tag is also created from javascript, so no help neither. The URL where the CSV data is retrieved from gives 401-Unauthorized http responses. So, it looks like the history data can't be reached. – bvdb Feb 04 '18 at 15:34
Seems like I need to pass along a consent cookie, because with the above code and printing the html I get stuff like ´
´ and not the requested page...
– Baked Inhalf Dec 20 '18 at 13:25

score 0 · Answer 2 · answered Nov 18 '13 at 05:56

The output file such as Apple Inc Stock Data at http://finance.yahoo.com/q;_ylt=Ag5D9mq4OAYIeUaL64JN7QYDyr0F;_ylc=X1MDMjE0MjQ3ODk0OARfcgMyBGZyA3VoM19maW5hbmNlX3dlYl9ncwRmcjIDc2EtZ3AEZ3ByaWQDBG5fZ3BzAzEwBG9yaWdpbgNmaW5hbmNlLnlhaG9vLmNvbQRwb3MDMQRwcXN0cgMEcXVlcnkDQUFQTCwEc2FjAzEEc2FvAzE-?p=http%3A%2F%2Ffinance.yahoo.com%2Fq%3Fs%3DAAPL%26ql%3D0&type=2button&fr=uh3_finance_web_gs&uhb=uhb2&s=AAPL is a CSV file. You could just read the content and parse by comma delimitor.

Jsoup Stock Quote Scraping Yahoo Finance

2 Answers2