How Can I Find a Specific Node By Using SelectSingleNode From HtmlAgilityPack

Question

Using the HtmlAgilityPack I am trying to obtain the text "9/30/2013" from a node on this website: http://www.nasdaq.com/symbol/goog/financials?query=income-statement&data=quarterly

Here is the HTML from the website

<div id="financials-iframe-wrap">
<br>
<div class="nextgen thin">
<div class="table-headtag">
<div style="float:left;">
<h3 style="color:#fff;">Quarterly Income Statement (values in 000's)</h3>
</div>
<div style="float:right;">
<h3><a id="quotes_content_left_hlswitchtype" href="http://www.nasdaq.com/symbol/goog/financials?query=income-statement" style="color:#fff;">Get Annual Data</a></h3>
</div>
</div>
<div style="clear:both"></div>
<table>
<tbody><tr class="tr_BG_Color">
<th class="th_No_BG">Quarter:</th>
<th style="text-align:left;">Trend</th>
<th>3rd</th>
<th>2nd</th>
<th>1st</th>
<th>4th</th>
</tr>
<tr class="tr_BG_Color">
<th class="th_No_BG">Quarter Ending:</th>
<th></th>
<th>9/30/2013</th>
<th>6/30/2013</th>
<th>3/31/2013</th>
<th>12/31/2012</th>
</tr>

And here is my code

Dim wreq As HttpWebRequest = WebRequest.Create("http://www.nasdaq.com/symbol/goog/financials?query=income-statement&data=quarterly")
    wreq.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5"
    wreq.Method = "get"
    Dim prox As IWebProxy = wreq.Proxy
    prox.Credentials = CredentialCache.DefaultCredentials
    Dim document As New HtmlAgilityPack.HtmlDocument
    Dim web As New HtmlAgilityPack.HtmlWeb
    web.UseCookies = True
    web.PreRequest = New HtmlAgilityPack.HtmlWeb.PreRequestHandler(AddressOf onPreReq)
    wreq.CookieContainer = cookies
    Dim res As HttpWebResponse = wreq.GetResponse()
    document.Load(res.GetResponseStream, True)
    Dim Page_Most_Recent_Quarter As Date = document.DocumentNode.SelectSingleNode("//*[@id='financials-iframe-wrap']/div/table//tr[2]/th[3]").InnerText

When my code reaches the last line I get this error Object reference not set to an instance of an object.

If I debug using Debug.WriteLine(document.DocumentNode.SelectSingleNode("//*[@id='financials-iframe-wrap']/div/table/tbody/tr[2]/th[3]")) a blank is returned.

What am I doing wrong?

Worked for me. Is the SelectSingleNode method returning an empty string, or null? — broke, Nov 05 '13 at 23:25
@broke How do I tell the difference between if the SelectSingleNode method returning an empty string, or null? — gromit1, Nov 07 '13 at 15:34
@broke You said that it worked for you. What value did it return when it worked for you? — gromit1, Nov 07 '13 at 15:34

broke · Answer 1 · 2013-11-08T15:52:24.980

First of all, why are you creating a HttpWebRequest object? Let the Html Agility Pack do the heavy lifting for you:

    Dim doc As New HtmlAgilityPack.HtmlDocument()

    Dim web As New HtmlAgilityPack.HtmlWeb()

    web.UseCookies = True

    doc = web.Load("http://www.nasdaq.com/symbol/goog/financials?query=income-statement&data=quarterly")

Once the HtmlDocument is loaded, we will extract the date:

        Dim dateNode As HtmlAgilityPack.HtmlNode = doc.DocumentNode.SelectSingleNode("//*[@id='financials-iframe-wrap']/div/table//tr[2]/th[3]")

        If dateNode IsNot Nothing Then
            Dim Page_Most_Recent_Quarter As Date = Convert.ToDateTime(dateNode.InnerHtml.Trim())
        End If

I tried this several times, and it works perfectly.

How Can I Find a Specific Node By Using SelectSingleNode From HtmlAgilityPack

1 Answers1