Undefined method error click for simple webscraper

Question

task :fetch_front => :environment do 
require 'rubygems'  
require 'nokogiri'
require 'open-uri'
require 'mechanize'
  agent = Mechanize.new

  agent.get("http://www.reddit.com/")

  agent.page.search("a.title").each do |thread|
   thread.click

  end
end

I am using mechanize to go into each reddit thread on the first page and return the top comment for each thread. The 'thread' block in the each method returns the link for each reddit thread. The problem is I am not sure how to click into the thread and return the top comment for each thread.

With my current code, it is returning an undefined method click error when I attempt to click into each thread to display the comments.

Answer 1

Call click on the agent and pass the thing you want to click on as an argument:

agent.click(thread)

Answer 2

Clicking on the thread title doesn't always take you to the thread comments page. To be sure, you must click on the link that have the thread comments count. I wrote (and tested), this code that will take you to every thread comments page.

require 'mechanize'
robot = Mechanize.new
response = robot.get('http://www.reddit.com/').parser

# Get the comments page link for every thread on the first page
response.css('#siteTable .thing a.comments').each do |thread_comments_link|
  comments_page = robot.get(thread_comments_link[:href]).parser
  # scrap comments page here
end

note: When you use mechanize you don't need to require nokogiri and open-uri. Nokogiri is a runtime dependencie of mechanize, so mechanize will require it for you, and open-uri is not necessary.