Get the href innertext with HtmlAgilityPack

Question

I am trying to create a news agent to get the news from the websites.so i have to use a html parser like HtmlAgilityPack .so here you ca see my code :

public async void parsing(string website)
{
    HttpClient http = new HttpClient();
    var response = await http.GetByteArrayAsync(website);
    String source = Encoding.GetEncoding("utf-8").GetString(response, 0, response.Length - 1);
    source = WebUtility.HtmlDecode(source);
    HtmlDocument resultat = new HtmlDocument();
    resultat.LoadHtml(source);
    List<HtmlNode> toftitle = resultat.DocumentNode.Descendants().Where
      (x => (x.Name == "div" && x.Attributes["class"] != null && x.Attributes["class"].Value.Contains("latest-news"))).ToList();
    var li = toftitle[0].Descendants("li").ToList();
    foreach (var item in li)
    {
        var link = item.Descendants("a").ToList()[0].GetAttributeValue("href", null);
        var img = item.Descendants("img").ToList()[0].GetAttributeValue("src", null);
    }
}

here is my html code that should be parsed :

<a href="/news?p_p_id=56_INSTANCE_tVzMoLp4zfGh&amp;_56_INSTANCE_tVzMoLp4zfGh_mode=news&amp;_56_INSTANCE_tVzMoLp4zfGh_newsId=3153832&amp;p_p_state=maximized">› پانل «بررسي سازوکارهاي تأمين منابع مالي براي توسعۀ فناوري» به‌عنوان پانل برتر پنجمين کنفرانس بين‌المللي و نهمين کنفرانس ملي مديريت فناوري معرفي شد</a>


<a href="/news?p_p_id=56_INSTANCE_tVzMoLp4zfGh&amp;_56_INSTANCE_tVzMoLp4zfGh_mode=news&amp;_56_INSTANCE_tVzMoLp4zfGh_newsId=3135970&amp;p_p_state=maximized">› فرآیند و فرم درخواست استفاده از تسهیلات حمایتی بلاعوض صندوق نوآوری و شکوفایی جهت حضور شرکت های دانش بنیان در جایزه ملی مدیریت فناوری و نوآوری</a>

So the problem is i can get the href link but not href value .I mean i can get the news url but not title پانل «بررسي سازوکارهاي تأمين منابع مالي براي توسعۀ فناوري» به‌عنوان پانل برتر پنجمين کنفرانس بين‌المللي و نهمين کنفرانس ملي مديريت فناوري معرفي شد.

How can i get that ?

There is only one value for href, the link. The "title" you want is the innertext of the a element. — Sami Kuhmonen, Jan 16 '16 at 08:26
@SamiKuhmonen i edit the post,i just added one of the li elements — Ehsan Akbar, Jan 16 '16 at 08:47

Uthman Rahimi · Answer 1 · 2016-01-16T09:56:06.273

4

you can use like this :

 HtmlAgilityPack.HtmlDocument doc = new HtmlAgilityPack.HtmlDocument();
 doc.LoadHtml(result);
 foreach (HtmlNode link in doc.DocumentNode.SelectNodes("a"))
        {
            string value = link.InnerText; // here you can get href value 
        }

edited Jan 16 '16 at 09:56

answered Jan 16 '16 at 08:55

Uthman Rahimi

708
5
22

score 3 · Accepted Answer · answered Jan 16 '16 at 09:37

3

I just should use this code to get the innertext of href :

string tistle = item.Descendants("a").ToList()[0].InnerText;

answered Jan 16 '16 at 09:37

Ehsan Akbar

6,977
19
96
180

score 1 · Answer 3 · answered Jan 16 '16 at 09:42

Your code can be cleaned up a bit to something like this :

List<HtmlNode> toftitle = resultat.DocumentNode
                                  .Descendants("div")
                                  .Where(x => GetAttributeValue("class","").Contains("latest-news"))
                                  .First();
foreach (HtmlNode item in toftitle.Descendants("li"))
{
    var link = item.Descendants("a").First();
    var url = link.GetAttributeValue("href", null); //get the link url
    var text = link.InnerText.Trim(); //get the link text

    var img = item.Descendants("img").First().GetAttributeValue("src", null);
}

Basically, you're supposed to use First() or FirstOrDefault() instead of ToList()[0] to get the first item of IEnumerable<T>.

Get the href innertext with HtmlAgilityPack

3 Answers3