Stack Overflow
Stack Overflow

Questions tagged [mechanize-ruby]

The Ruby library for automating interaction with websites.

The Mechanize library is used for automating interaction with websites. Mechanize automatically stores and sends cookies, follows redirects, can follow links, and submit forms. Form fields can be populated and submitted. Mechanize also keeps track of the sites that you have visited as a history.

193 questions
0
votes
1 answer

Ruby Mechanize: verifying a "select" element value

I'm attempting to verify a value for a drop down using ruby's mechanize I've got this pretty printed form: Before: #
carl crott
  • 753
  • 1
  • 9
  • 21
0
votes
2 answers

Twitter API vs Web scraping?

To begin with I want to get all the followers of my twitter account. I did a little research and found that we can do web scraping with Ruby on rails using : Nokogiri or Mechanize gems. I also got a css selector to use for the web scraping. Now the…
Rubyuser
  • 79
  • 1
  • 1
  • 10
0
votes
2 answers

Why does Mechanize raise "undefined method 'any?'" when it sees bad OAuth credentials?

I'm testing some Ruby code that uses Mechanize under the covers. I found that when I pass a bad OAuth token in a request, Mechanize throws an unexpected exception. I make my call with the bad token, and the call raises the following…
Bruce
  • 8,202
  • 6
  • 37
  • 49
0
votes
1 answer

Recover net::ERR_CONNECTION_SET Mechanize Ruby

I'm creating a web crawler using Mechanize for ruby. I'll be running batches of 200k at a time and I want to be able to set an instance variable that the site is not valid and move on with the next site when the get request return an error. For…
Luis D Urraca
  • 2,024
  • 4
  • 24
  • 46
0
votes
1 answer

How to find and click an object based on title?

I want to click a html object based on title
Radek
  • 13,813
  • 52
  • 161
  • 255
0
votes
1 answer

Trouble with mechanize and submitting a text area

So I'm trying to submit a simple form that has one text area that looks similar to this "Lots of stuff that doesn't matter a couple of lines later agent => A5656,,5665" What I'm trying to do is take out the line with the agent. By the way, there…
David Hahn
  • 740
  • 9
  • 26
0
votes
1 answer

Ruby webscrape script for GoDaddy

I'm new to Ruby and for my first scripting assignment, I've been asked to write a web scraping script to grab elements of our DNS listings from GoDaddy. Having issues with scraping the links and then I need to follow the links. I need to get the…
Lynn
  • 559
  • 1
  • 6
  • 22
0
votes
1 answer

Trying to find image url via xpath using Mechanize

I am trying to find the image xpath for the following page: http://www.spoonsisters.com/product/1032000/38710.html I can view the image_url via my browser, however when I try finding it via Mechanize: page =…
Yogzzz
  • 2,735
  • 5
  • 36
  • 56
0
votes
1 answer

Ruby Mechanize code converter not found

I have tried a ton of stuff for instance: Mechanize on Ruby 1.9.3 encoding issue and How do I convert from a Mechanize::File object to a Mechanize::Page object? But the error I keep getting is…
David Hahn
  • 740
  • 9
  • 26
0
votes
1 answer

Click on input tag, not in form

Im using mechanize to scrape a few pages. Pagination is implemented by a javascript post, and the pagination links are actually input buttons. These are not included in a form. Any idea how I could trigger a click on these? Im still working on the…
zsquare
  • 9,916
  • 6
  • 53
  • 87
0
votes
1 answer

Parsing with Ruby Mechanize

Im trying to parse a website using the Mechanize Gem. So far this is what I have: page = agent.get("http://www.greatgiftsformen.com/price-range-under-c-131_142.html?page=all") page.parser.xpath('//tr[(((count(preceding-sibling::*) + 1) = 2) and…
Yogzzz
  • 2,735
  • 5
  • 36
  • 56
-1
votes
1 answer

select elements and click link if condition

How to write a code that will click links on a webpage with specified conditions: The script should look into each div.table-row and check if list element (div.domains ul li) in the div contains certain string (somedomain.com), and click the link…
all jazz
  • 2,007
  • 2
  • 21
  • 37
-2
votes
1 answer

Extract data from website

I want to extract properties data from the website, I have used mechanize for getting the data from few websites, I have tried mechanize in this website i got all repeated results of the first page results. But in this website, all the results…
Ashwin Yaprala
  • 2,737
  • 2
  • 24
  • 56
1 2 3
…
12
13