7

I am using mechanize to scrap some web pages.

  • I need to know what are mechanize limitations? What mechanize can not do?
  • Can it execute javascripts embedded in the web page?
  • Can I use it to call javascript functions? I don't think it can. I think Watir can.
  • What are the differences between it and watir?
Ripon Al Wasim
  • 36,924
  • 42
  • 155
  • 176
Mahmoud Khaled
  • 6,226
  • 6
  • 37
  • 42

1 Answers1

13

Mechanize can do a lot. It uses net/http so whatever you can do with net/http you can do with mechanize. Although it supports much more as per their description :

The Mechanize library is used for automating interaction with websites. Mechanize automatically stores and sends cookies, follows redirects, can follow links, and submit forms. Form fields can be populated and submitted. Mechanize also keeps track of the sites that you have visited as a history.

Check out this link for some information on using javascript with mechanize: here

It would be much easier to tell you if mechanize support a specific function/task instead of going through everything. What are you looking to do exactly ?

Javascript is the one thing mechanize can't do. The one thing it does support most of the time is displaying Javascript links. ie using page.links.each {|link| puts link.text} will also display Javascript, but you won't be able to click/select them.

In simple terms Watir does support Javascript. It's actually your browser that supports javascript and Watir controls the browser.

Watir runs a real browser(FF,Chrome,IE) and programmatically controls that browser. It acts exactly like a user would when accessing a website. This is what enables you to use javascript. Watir only controls the browser and the browser is the one sending requests and getting responses and rendering/processing it all. You are limited by the speed of the browser you use.

Mechanize on the other hand acts like its own 'browser' and is much faster than Watir, becomes it does not render pages. It talks directly with the server, and processes the raw response. Mechanize is limited by your connection speed.

Watir would be used over Mechanize when you need to watch and see what's happening, use javascript, or do anything GUI related. Mechanize is much faster and is good for testing the actual structure of the website. (testing links/logins/etc)

Kassym Dorsel
  • 4,773
  • 1
  • 25
  • 52
  • 1
    Technically, Watir only supports IE. To do FireFox (current versions), Chrome, Opera, or operate headless, you need to use Watir-Webdriver. (Same basic Watir API, but uses webdriver to control the browsers) There is also an .execute_script method, but it is generally frowned upon since that isn't something a user can do, however in some cases it's the easiest way around complicated event driven controls – Chuck van der Linden Nov 02 '11 at 22:23
  • 1
    Can watir work over linux server without "display"? how will it open browser without OS display? – Mahmoud Khaled May 15 '12 at 10:06
  • 1
    You would need to go headless, I've never tried it and wouldn't know how to do it. Perhaps look at the Mechanize gem instead. – Kassym Dorsel May 15 '12 at 21:10