Stack Overflow
Stack Overflow

Questions tagged [mechanize-ruby]

The Ruby library for automating interaction with websites.

The Mechanize library is used for automating interaction with websites. Mechanize automatically stores and sends cookies, follows redirects, can follow links, and submit forms. Form fields can be populated and submitted. Mechanize also keeps track of the sites that you have visited as a history.

193 questions
2
votes
1 answer

How do I convert a Nokogiri statement into Mechanize for screen scraping?

I'm trying to use Mechanize to scape some tags from a page. I've used Nokogiri successfully to scrape them before, but now I'm trying to combine them into a wider Mechanize class. Here is the Nokogiri statement: page =…
Adam
  • 2,917
  • 5
  • 26
  • 27
2
votes
1 answer

why we need user_agent_alias with mechanize object?

I just wondering for some informations about mechanize and found the below code from Internet: require 'mechanize' require 'logger' agent = Mechanize.new agent.user_agent_alias = 'Windows IE 9' agent.follow_meta_refresh = true agent.log =…
Arup Rakshit
  • 116,827
  • 30
  • 260
  • 317
2
votes
1 answer

Getting error "getaddrinfo: No such host is known. (Socke tError)" with mechanize gem

I tried the below code: require 'mechanize' agent = Mechanize.new{|a| a.ssl_version, a.verify_mode = 'SSLv3', OpenSSL::SSL::VERIFY_NONE} page = agent.get "https://gegsltraining.aravo.com/" page=page.link_with(:dom_class =>…
Arup Rakshit
  • 116,827
  • 30
  • 260
  • 317
2
votes
2 answers

Can I use Mechanize to download files with .docx and .xlsx,.txt extensions?

I browsed the documentation Mechanize. The below is for .pdf only. require 'mechanize' agent = Mechanize.new agent.pluggable_parser.pdf = Mechanize::FileSaver agent.get 'http://example.com/foo.pdf' But can I also download .docx,.xlsx,.txt file…
DoLoveSky
  • 777
  • 1
  • 6
  • 17
2
votes
2 answers

Unexplained Inconsistency when Downloading an XLS file with Ruby Mechanize after redirect

I have a script that visits fcc.gov, then clicks a link which triggers a download: require "mechanize" docket_number = "12-268" #"96-128" url = "http://apps.fcc.gov/ecfs/comment_search/execute?proceeding=#{docket_number}" agent =…
user94154
  • 16,176
  • 20
  • 77
  • 116
2
votes
1 answer

How can I download a file using its generated filename with ruby and mechanize?

I'm attempting to download files from a website that uses a CDN for distribution. The URLs on the download page all end with file.pdf but clicking on the link in a browser results in the download of a file with a descriptive file name (e.g.…
JP.
  • 5,536
  • 7
  • 58
  • 100
2
votes
2 answers

Ruby Mechanize issues with http authentication

I'm having issues getting around websites that use http authentication, I have a list of sites which I do some scrapping on but some of these have http authentication on them. I'm not looking to get the content of those sites I want to be able to be…
Luis D Urraca
  • 2,024
  • 4
  • 24
  • 46
2
votes
1 answer

When submitting a login form using mechanize-ruby, Can I use variables to represent field names?

I ran into a problem when using Mechanize to submit a login form. For example, if I need to log into bitbucket: a = Mechanize.new a.get('https://bitbucket.org/') do |page| login_page = a.click(page.link_with(text: 'Log In')) my_page =…
blacktulip
  • 198
  • 2
  • 9
2
votes
2 answers

Ruby:: How to search hidden elements with mechanize

I am trying to get hidden field with mechanize in ruby and trying to click on it. agent = Mechanize.new agent.get('http://www.example.com/') agent.page.link_with(:text => "More Links...") But this gives me: => nil Actually, I want to click on…
sagar junnarkar
  • 1,240
  • 2
  • 10
  • 18
2
votes
2 answers

How do I convert from a Mechanize::File object to a Mechanize::Page object?

I have a page that logs into a form. After logging in there are a few redirects. The first one looks like this: #"no-cache=\"set-cookie\"",…
user1198316
  • 267
  • 2
  • 3
  • 12
2
votes
1 answer

Ruby::Mechanize::cookie_jar

I have a site that does four different re-directs and doesn't seem to always pass the cookies along. So I've researched here and tried a few of the solutions like: creating a temp cookie_jar and assigning that each time. I've also attempted to…
user1198316
  • 267
  • 2
  • 3
  • 12
1
vote
0 answers

Rails mechanize can't access page that requires logging in

I am trying click a link that appears after I've logged in with Rails + Mechanize, but it returns "Cookie should be enabled". I've checked the cookie in the my Mechanize object (mechanize.cookie_jar.save_as('cookies.yml')) and it exists and is the…
Dzmitry
  • 749
  • 2
  • 11
  • 25
1
vote
0 answers

How to pass cookie in header of Mechanize get method?

hi I am trying to access the page through mechanize get method.The page takes cookie of the previous page post request to give response. Can anyone tell me how to pass the cookie in the header of get method of Mechanize? I am trying the approach…
zargham shahid
  • 11
  • 2
1
vote
1 answer

Installing mechanize gem on Mac OS X 10.4.11 gives 'Failed to build gem native extension'

I'm trying to install mechanize gem on a MAC OS X but I keep getting the following error : ERROR: Error installing mechanize: ERROR: Failed to build gem native extension. /usr/local/bin/ruby extconf.rb install mechanize checking for…
dragosh
1
vote
1 answer

Handling javascript onchange events with Mechanize rubygem

I have been trying to write a script that fetches results from my university website. Someone suggested that I use Mechanize and it does look really promising. In order to get the result, one has to first enter the roll number and then select the…
nikhil
  • 8,925
  • 21
  • 62
  • 102
1 2 3
…
12 13