Stack Overflow
Stack Overflow

Questions tagged [mechanize-ruby]

The Ruby library for automating interaction with websites.

The Mechanize library is used for automating interaction with websites. Mechanize automatically stores and sends cookies, follows redirects, can follow links, and submit forms. Form fields can be populated and submitted. Mechanize also keeps track of the sites that you have visited as a history.

193 questions
0
votes
0 answers

Why does Mechanize gets different results than a human search?

I'm using the following code: require 'rubygems' require 'mechanize' require 'nokogiri' require 'open-uri' require 'logger' require 'slowweb' SlowWeb.limit('linkedin.com', 1, 10) #create agent agent = Mechanize.new { |agent| …
user1222303
  • 23
  • 3
0
votes
1 answer

What's wrong with this Ruby Mechanize login to stackoverflow script?

What is wrong with this? This gets me past the first login page (I think), but I appear to remain logged out. require 'rubygems' require 'mechanize' #create agent agent = Mechanize.new { |agent| agent.user_agent_alias = 'Mac Safari…
user1222303
  • 23
  • 3
0
votes
2 answers

Why does this ruby mechanize code throw undefined method?

I'm picking up ruby mechanize & getting tripped up from the start... Why does this code: #!/usr/bin/ruby env require 'rubygems' require 'mechanize' agent = Mechanize.new page = agent.get('http://linkedin.com/') #pp page …
user1222303
  • 23
  • 3
0
votes
1 answer

Rails : How to click on a button using Mechanize gem?

I am trying to click on youtube's Load more button in the user's playlist section using Mechanize gem . But the problem is , i din't find any ways of clicking on a button using mechanize gem in my Rails application. The html code of button which i…
huzefa biyawarwala
  • 637
  • 7
  • 24
0
votes
1 answer

Not able to login into rottentomatoes.com using mechanize

I am using following code :- require 'rubygems' require 'mechanize' agent = Mechanize.new agent.get("https://www.rottentomatoes.com/user/account/login/") do |login_page| inside_page = login_page.form_with(:action =>…
vivek pandey
  • 1
0
votes
1 answer

How to get text from list items with Mechanize?

  • "Car 1"
  • "Car 2"
  • "Car 3"
  • "Car 4"
I want strip the text from each list item with mechanize and print…
user3131148
  • 343
  • 1
  • 5
  • 18
0
votes
1 answer

Mechanize and Nokogiri printing to terminal instead of file

No clue on this one.. Its acting weird, and the response I want is getting put to the terminal instead of the file.. But I never asked the results to be put to the terminal. Odd.. How can I get the results from Mechanize / Nokogiri printed to a…
ToddT
  • 3,084
  • 4
  • 39
  • 83
0
votes
1 answer

ruby mechanize proxy retry until connected

What I want to do is this I have a proxy list with ip|port Now Mechanize is using those proxy and I have set open_timeout = 20 read_timeout = 20 What I want it todo is to retry, but it will retry with the same proxy even if I have "proxies =…
Cyrus Zei
  • 2,589
  • 1
  • 27
  • 37
0
votes
1 answer

Mechanize form submission

I have a website that I am attempting to scrape using Mechanize. When I submit the form, the form is submitted with an URL of the following format : https://www.website.com/Login/Options?returnURL=some_form_options (If I enter that URL in the…
harvey
  • 299
  • 1
  • 8
0
votes
2 answers

Require 'mechanize' gem issue

I can't seem to be able to use mechanize in irb. Here is the error message returned when attempting to : require 'mechanize' 2.2.0 :003 > require 'mechanize' LoadError: cannot load such file -- mechanize from…
harvey
  • 299
  • 1
  • 8
0
votes
1 answer

weird error when installing mechainze gem on mac os

I'm trying to install the mechainze gem on my mac os, however when I typed gem install mechanize, after a few seconds, it poped out: Building native extensions. This could take a while... ERROR: Error installing mechanize: ERROR: Failed to…
milodky
  • 443
  • 2
  • 7
  • 18
0
votes
1 answer

Dashing (Ruby) Nokogiri LoadError

I've been working on a dashboard on the Dashing framework, and I'm currently trying to make a little crawler to collect specific data on Jenkins-CI, and pass it to the Number widget. Here's the crawler (it's just a stub, it counts the number of "p"…
Filipe Gorges Reuwsaat
  • 856
  • 1
  • 9
  • 23
0
votes
1 answer

how to extract these specific links usuing mechanize in Ruby?

I've been trying but I cant get these specific links on this page: http://www.windowsphone.com/en-us/store/top-free-apps I want to get each one of the links on the left side of this page, entertainment for example, but I cant find the right…
Amin
  • 1
  • 1
0
votes
1 answer

Best scraper/crawler needed for multipage forms (Nokogiri, Scrapy, other?)

I've read that Nokogiri/Mechanize (Ruby) for example are not good at traversing multiple pages, but may be better with sites that use Ajax. The sites I want to scrape are multi-page forms, with some ajax overlays. Speed is important. These sites…
hippypink
  • 17
  • 2
0
votes
0 answers

mechanize ruby - how do do wait for a javascript based update to a page?

mechanize ruby (i.e. http://docs.seattlerb.org/mechanize/) - how do do wait for a javascript based update to a page? I have a site that requires login, but then takes many seconds to login, but after which it's really handled (it appears) as…
Greg
  • 34,042
  • 79
  • 253
  • 454
1 2 3
…
12 13