Stack Overflow
Stack Overflow

Questions tagged [mechanize-ruby]

The Ruby library for automating interaction with websites.

The Mechanize library is used for automating interaction with websites. Mechanize automatically stores and sends cookies, follows redirects, can follow links, and submit forms. Form fields can be populated and submitted. Mechanize also keeps track of the sites that you have visited as a history.

193 questions
0
votes
2 answers

SSLv3 read server certificate error while using Mechanize to log into to flicker.com

I am trying to execute following code in Windows 7 machine using Mechanize and Ruby. require 'mechanize' a = Mechanize.new { |agent| # Flickr refreshes after login agent.follow_meta_refresh = true } a.get('https://www.flickr.com/') do…
Chuchoo
  • 823
  • 2
  • 17
  • 36
0
votes
3 answers

NoMethodError in from Mechanize

running this code with mecahnize 2.7.3 and ruby 2.3.0dev: require 'mechanize' agent = Mechanize.new agent.keep_alive = false agent.open_timeout = 2 agent.read_timeout = 2 agent.ignore_bad_chunking = true agent.gzip_enabled = false url =…
pinpox
  • 179
  • 2
  • 10
0
votes
1 answer

Mechanize: Unable to redirect to final destination

I am using mechanize to try to login to a site. However when logging in with mechanize, I cant seem to get to final destination. Below is the example. Mechanize go to (1) https://app.abc.com/users/login If login successful, webpage will momentarily…
chickensmitten
  • 477
  • 6
  • 16
0
votes
1 answer

Ruby Mechanize Set Form Element Values

I thought I had Mechanize locked down with regards to setting form field values, but my submission isn't taking. CCCform is my form object, so my field set is this for the field in question: CCCform['searchCriteriaBean.conceptOperator'] =…
Rich_F
  • 1,830
  • 3
  • 24
  • 45
0
votes
1 answer

Mechanize Traverse & Parse

I'm trying to walk through a hitlist of pages found by Mechanize. Upon a search, which is working just fine, I get a hitlist with 10 records per page. A bottom navigation system takes me to the record count > 10. Display 10 per page. So 53…
Rich_F
  • 1,830
  • 3
  • 24
  • 45
0
votes
1 answer

Ruby Mechanize Grab HTML Behind Links in Array

I'm using Mechanize to grab a bunch of pages behind links. On page A, there's a bunch of companies that have a link called "[complete profile]", behind which is full html I want to grab. There are 10 of these on page A. I can't seem to traverse…
Rich_F
  • 1,830
  • 3
  • 24
  • 45
0
votes
2 answers

Undefined method error click for simple webscraper

task :fetch_front => :environment do require 'rubygems' require 'nokogiri' require 'open-uri' require 'mechanize' agent = Mechanize.new agent.get("http://www.reddit.com/") agent.page.search("a.title").each do |thread| thread.click …
Jon Nnnyguyen
  • 1
0
votes
1 answer

I cannot edit a field with mechanize

I am developing a Rails application and I am using mechanize to automate the changing of a profile. Currently I am trying to change a field date that has some sort of mask, as it only displays the format. (dd/mm/yyyy). It has two fields:…
Wiggin
  • 329
  • 1
  • 3
  • 17
0
votes
0 answers

Nokogiri without DIV

I am trying to scrape information from an HTML table. There are multiple tables on the page. Before each table there is a paragraph with text. I want to key off of this text field ("CONSOLIDATED" text in the pastebin below) to identify the table…
sizzle
  • 2,222
  • 2
  • 21
  • 32
0
votes
1 answer

Add unnamed field to form. Mechanize Ruby

I am struggling to add fields to a form, because Mechanize doesn't detect the fields. I have 3 forms and I managed to select the correct form. Entering p site.forms gives the following: => [#
Basher52
  • 80
  • 7
0
votes
1 answer

Extract Email Address with Mechanize

I am trying to return email addresses from a website using mechanize. I am very easily able to determine whether or not the "@" symbol is found on the page by using the code below. However, I would like to return the characters surrounding the @…
Brandon
  • 1,701
  • 3
  • 16
  • 26
0
votes
1 answer

Find and act on links based on value in a cell with Mechanize in Rails

I am trying to act on links found in a specific row in an HTML table. The table has a column for a date and a column for city names which act as links. I need to find row with specific date and then cycle through the links found from the next cell…
thepanu
  • 121
  • 1
  • 1
  • 12
0
votes
1 answer

Crawl data using ruby mechanize

I am crawling data from http://www.mca.gov.in/DCAPortalWeb/dca/MyMCALogin.do?method=setDefaultProperty&mode=53 Below is the code I have tried : uri = "http://www.mca.gov.in/DCAPortalWeb/dca/MyMCALogin.do?method=setDefaultProperty&mode=53" …
Vivek Soni
  • 3
  • 3
0
votes
1 answer

Ruby Mechanize Stops Working while in Each Do Loop

I am using a mechanize Ruby script to loop through about 1,000 records in a tab delimited file. Everything works as expected until i reach about 300 records. Once I get to about 300 records, my script keeps calling rescue on every attempt and…
Brandon
  • 1,701
  • 3
  • 16
  • 26
0
votes
1 answer

Skip large pages in Mechanize (Ruby)

I'm trying to skip processing a few large pages (some over 10MB) scattered in a result set, as Mechanize (version 2.7.3) crawls an array of links. Unfortunately I can't find a 'content-length' property or a similar indicator. The…
odlp
  • 4,984
  • 2
  • 34
  • 45
1 2 3
…
12 13