Extract data from website

Question

I want to extract properties data from the website,

I have used mechanize for getting the data from few websites, I have tried mechanize in this website i got all repeated results of the first page results.

But in this website, all the results calls are based on ajax calls,

How can i grab the data from this type of websites, where they using ajax, and in some ajax call itself encrypted.

Thanks in advance

might be you have to fire ajax request with required params or click on page no based on position — Amar, Jan 07 '13 at 09:41

score 0 · Answer 1 · answered Jan 07 '13 at 20:34

You can't get the result of an Ajax request using Mechanize, or a normal HTTP request, during the first load of a page. You have to make subsequent requests based on parameters in that page, to their Ajax host, with the appropriate values, the same ones the browser would use after loading the page.

This requires either using something like Watir, so you can script it and then retrieve the data received by the Ajax request, or by analyzing the embedded JavaScript to figure out the parameters being used by the Ajax request and spoofing their system with another Mechanize or HTTP request.

Extract data from website

1 Answers1