0

I am using Mechanize gem to parse html content.

Firing this on terminal,

 agent.get("http://www.example.com/").search(".sidebar-deal-excerpt").first

gives me this:

 <Nokogiri::XML::Element:0x3fcdb3add700 name="div" attributes=[#<Nokogiri::XML::Attr:0x3fcdb3add570 name="class" value="sidebar-deal-excerpt">] children=[#<Nokogiri::XML::Text:0x3fcdb3add0ac "\r\n\t\t\t\t\t\t\t\t\t\t\t\t\t">, #<Nokogiri::XML::Element:0x3fcdb3adcfd0 name="a" attributes=[#<Nokogiri::XML::Attr:0x3fcdb3adcf6c name="href" value="http://www.example.com/getaway-8774-yatra-september-deal">] children=[#<Nokogiri::XML::Text:0x3fcdb3adcaa8 " ">, #<Nokogiri::XML::Element:0x3fcdb3adc9cc name="span" attributes=[#<Nokogiri::XML::Attr:0x3fcdb3adc968 name="style" value="color: #000">] children=[#<Nokogiri::XML::Text:0x3fcdb3dd6b50 " international holiday package bookings at Yatra.com ">]>, #<Nokogiri::XML::Text:0x3fcdb3dd5f5c " ">]>, #<Nokogiri::XML::Text:0x3fcdb3dd5a98 "\r\n\t\t\t\t\t\t\t\t\t\t\t\t">]>

Firing this on terminal

agent.get("http://www.example.com/").search(".sidebar-deal-excerpt").first.children

gives me this:

 <Nokogiri::XML::Text:0x3fcdb399c3dc "\r\n\t\t\t\t\t\t\t\t\t\t\t\t\t">, #<Nokogiri::XML::Element:0x3fcdb399c300 name="a" attributes=[#<Nokogiri::XML::Attr:0x3fcdb399c29c name="href" value="http://www.example.com/getaway-8774-yatra-september-deal">] children=[#<Nokogiri::XML::Text:0x3fcdb399bdd8 " ">, #<Nokogiri::XML::Element:0x3fcdb399bcfc name="span" attributes=[#<Nokogiri::XML::Attr:0x3fcdb399bc98 name="style" value="color: #000">] children=[#<Nokogiri::XML::Text:0x3fcdb399b7d4 "international holiday package bookings at Yatra.com ">]>, #<Nokogiri::XML::Text:0x3fcdb3dd8770 " ">]>, #<Nokogiri::XML::Text:0x3fcdb3dd6df8 "\r\n\t\t\t\t\t\t\t\t\t\t\t\t">]

The thing is, I want to fetch url from this. I want to get this attribute value 

value="http://www.example.com/getaway-8774-yatra-september-deal"

ie 

   http://www.example.com/getaway-8774-yatra-september-deal

How can I parse this?

halfer
  • 19,824
  • 17
  • 99
  • 186
Mohit Jain
  • 43,139
  • 57
  • 169
  • 274

1 Answers1

1

How about: (untested)

agent = Mechanize.new
page = agent.get("http://www.snapdeal.com/")
page = agent.get("http://www.snapdeal.com/") # yeah, you've got to call it twice for this site
urls = page.search(".sidebar-deal-excerpt a").collect{|a| a.attributes["href"].value }
Candide
  • 30,469
  • 8
  • 53
  • 60
  • no its not working.. here is the website i want to parse. Sidebar for snapdeal.com – Mohit Jain Sep 14 '11 at 18:46
  • See code above. to get the value, you have to use a.attributes["href"].value – Candide Sep 14 '11 at 19:00
  • No, I think the first request sets a cookie. The first time I visited the site I got a drop down. The second time, the cookies are sent over, and the site responds differently. – Candide Sep 14 '11 at 19:16
  • can u help me with this question http://stackoverflow.com/questions/7422799/how-to-trigger-click-event-using-mechanize-and-then-get-the-ajax-content – Mohit Jain Sep 14 '11 at 22:24