can't find stock price in yahoo finance using selenium webdriver

Question

I've been trying to find a way to pull stock data from yahoo finance for the past two days. I've used selenium webdriver before and I find it very useful, however I can't seem to get it to locate the price for stocks. The html is rather complicated in finance.yahoo.com but I think you can isolate the tags so my code looks something like this:

public void writeSheet() throws WriteException, Exception{
    int r = 0;
    for(String s : listOfFunds){
        Label stockLabel = new Label(0,r,s);
        Number stockPrice = new Number(1,r,0.00);
        driver.get("http://finance.yahoo.com/q?s=" + s + "%2C+&q1=q");
        Thread.sleep(200);
        WebElement price = driver.findElement(By.tagName("span"));
        stockPrice.setValue(Double.parseDouble(price.findElement(By.id("yfs_184_" + s.toLowerCase())).getText()));
        id=\"yfs_184_" + s.toLowerCase() + "\""));
        sourceSheet.addCell(stockLabel);
        sourceSheet.addCell(stockPrice);
        r++;
    }
    sourceBook.write();
    sourceBook.close();
}

I'm really just sort of checking to see I didn't miss anything stupid or if I'm searching by the tag name. The one HTML line I'm looking for looks like this:

<span id="yfs_184_" + insert symbol to lowercase here + ">stock price</span>

what is the error you are getting? – Abhijeet Vaikar Mar 13 '14 at 05:26 — Abhijeet Vaikar, Mar 13 '14 at 05:26

Abhijeet Vaikar · Accepted Answer · 2014-03-13T06:11:13.240

0

I tried this and it worked very well.

WebElement stockpriceText = driver.findElement(By.cssSelector("span[id=\"yfs_l84_ctsh\"]"));
String price = stockpriceText.getText();

Output: 51.59

I believe your locating strategy is a bit wrong. You are trying to first find a span in:

WebElement price = driver.findElement(By.tagName("span"));

which could be the problem. WebDriver will find the very first <span> element on the page. It could be some other span in the DOM.
Plus when you are executing this:

price.findElement(By.id("yfs_184_" + s.toLowerCase())).getText();

That time you are trying to find another element inside the span element which will not work.

edited Mar 13 '14 at 06:11

answered Mar 13 '14 at 06:02

Abhijeet Vaikar

1,578
4
27
50

@jDave1984: Consider accepting or upvoting the answer if it works for you. – Abhijeet Vaikar Mar 13 '14 at 13:00
I will try this when I get home. I had tried loading all the elements into a List but that didn't work. I didn't even think about the cssSelector. I will let you know how it turns out. Thanks! – jDave1984 Mar 13 '14 at 13:09
I tried it out... I'm getting the NoSuchElementException. Not sure what I'm doing wrong – jDave1984 Mar 13 '14 at 23:07
Yeah it's not working. That css isn't showing anywhere in the source for any yahoo finance page. Am I not loading a page correctly? – jDave1984 Mar 13 '14 at 23:29
Are you trying it using your old way or you tried my approach? css and css selector are different things. You should not be looking out the locator `span[id=\"yfs_l84_ctsh\"]` in the **css** source of the page. – Abhijeet Vaikar Mar 14 '14 at 04:47
.........No both wound up working, now... I have the "l" as a 1 (number one). Changed that and that part is working now. Hooray for L's looking like ones! – jDave1984 Mar 14 '14 at 15:56

can't find stock price in yahoo finance using selenium webdriver

1 Answers1