cURL works but mechanize doesn't

Question

I am trying to scrape my university web site using ruby mechanize. This my ruby script;

require 'mechanize'
agent = Mechanize.new
agent.get('https://kampus.izu.edu.tr')

This script doesn't return response. I need to see login page but the response is different. I also tried it with cURL like this;

curl https://kampus.izu.edu.tr

This works and return the login page. What am I missing?

@pguardiario i have same environment. can you add the response and ruby script ? — Bilal Ünal, Jan 09 '18 at 16:23
I just typed those 3 lines into irb and I got a good response. — pguardiario, Jan 10 '18 at 00:21

score 0 · Answer 1 · answered Aug 08 '19 at 22:10

Make sure that you are storing the output of agent.get(). From your example, I don't see how you would be using/printing the response of this request.

Try this:

require 'mechanize'
agent = Mechanize.new 
page = agent.get("https://kampus.izu.edu.tr")
puts page.body

The .get() method returns a Mechanize::Page object that you can call other methods on, such as .css(), to select elements by css selectors. Check out the documentation here

cURL works but mechanize doesn't

1 Answers1