1

I'm trying to scrape a website and do I need to use Javascript. I am using Python 2.7 and Spynner. This is the script:

   <a href="javascript://" onclick="TranslationPopUpTestimonialQuery('Language_ALL', '',       '5', '10', 'date-of-visit')">6</a>

This is my code in Python:

   import spynner
   browser = spynner.Browser()
   browser.load("http://www.zoover.it/italia/lazio/roma/galileo/hotel")
   page1 = browser._get_html()
   browser.runjs("TranslationPopUpTestimonialQuery('Language_ALL', '', '8', '10', 'date-of-visit')")
   browser.wait_load()
   page2 = browser._get_html()

Something does not work as get_html does not give me the desired html page...

This the first url: http://www.zoover.it/italia/lazio/roma/galileo/hotel. I load this page with browser.load() and i get the HTML. After i try to go to page 2 simulating a click with browser.runjs(). I call the javascript on the onClick. Then If i call browser._get_html() another time the HTML is ever the same. Can anyone help me? thanks

RoverDar
  • 441
  • 2
  • 12
  • 32
  • Please be more specific about the problem that you are encountering. – Marcin Sep 18 '12 at 18:19
  • I have read several pages from a url starting (eg: www.mypage.com). To move from one page to another i must click on the link with the number (eg: 2.3.4.5...). The link activate the javascript. Mi problem is: my code loads only the first page and not the other. (Sorry for my english) – RoverDar Sep 18 '12 at 18:54
  • Put a fuller description, together with code and output demonstrating the problem, in your question. Please use real URLs. http://sscce.org/ – Marcin Sep 18 '12 at 19:01
  • Excuse me. This the first url: http://www.zoover.it/italia/lazio/roma/galileo/hotel. I load this page with browser.load() and i get the HTML. After i try to go to page 2 simulating a click with browser.runjs(). I call the javascript on the onClick. Then If i call browser._get_html() another time the HTML is ever the same. – RoverDar Sep 18 '12 at 19:19
  • This is all great information. *Put it in your question* – Marcin Sep 18 '12 at 19:22

1 Answers1

0

It the html you want is in a popup, you ll have to switch spynner frames.

What spynner gives you is always the active frame html, popups open in another frames.

Spynner is an interface to qtwebwekit and qtwebkit use that : http://doc.qt.digia.com/4.7-snapshot/qwebframe.html

You can see that there is "childframes", that you can introspect with self.webframe.

kiorky
  • 299
  • 1
  • 8
  • Thank you very much for the information. Ok I tray to see. It's no easy for me .. can you explain wjth an exemple? Thanks again – RoverDar Oct 09 '12 at 14:51