Preventing timeout when connecting to a URL

Question

I want to see the time taken to access a url using Benchmark in the code below. I also tried to do the same thing without benchmark. That is, get time at start of test and end of test, subtract the two to get the time. Both methods end in the same timeout error.

require 'open-uri'
require 'benchmark'

response = nil
puts "opening website with benchmark..."
puts Benchmark.measure{
  response = open('http://mywebsite.com')
}

puts "Done !"
status = response.status
puts status

Error:

opening website with benchmark...
C:/ruby/lib/ruby/1.8/timeout.rb:64:in `rbuf_fill': execution expired (Timeout::Error)
    from C:/ruby/lib/ruby/1.8/net/protocol.rb:134:in `rbuf_fill'
    from C:/ruby/lib/ruby/1.8/net/protocol.rb:116:in `readuntil'
    from C:/ruby/lib/ruby/1.8/net/protocol.rb:126:in `readline'
    from C:/ruby/lib/ruby/1.8/net/http.rb:2028:in `read_status_line'
    from C:/ruby/lib/ruby/1.8/net/http.rb:2017:in `read_new'
    from C:/ruby/lib/ruby/1.8/net/http.rb:1051:in `request'
    from C:/ruby/lib/ruby/1.8/open-uri.rb:248:in `open_http'
    from C:/ruby/lib/ruby/1.8/net/http.rb:543:in `start'
    from C:/ruby/lib/ruby/1.8/open-uri.rb:242:in `open_http'
    from C:/ruby/lib/ruby/1.8/open-uri.rb:616:in `buffer_open'
    from C:/ruby/lib/ruby/1.8/open-uri.rb:164:in `open_loop'
    from C:/ruby/lib/ruby/1.8/open-uri.rb:162:in `catch'
    from C:/ruby/lib/ruby/1.8/open-uri.rb:162:in `open_loop'
    from C:/ruby/lib/ruby/1.8/open-uri.rb:132:in `open_uri'
    from C:/ruby/lib/ruby/1.8/open-uri.rb:518:in `open'
    from C:/ruby/lib/ruby/1.8/open-uri.rb:30:in `open'
    from C:/code/test.rb:7
    from C:/ruby/lib/ruby/1.8/benchmark.rb:293:in `measure'
    from C:/code/test.rb:6

When I try to connect to this URL in my browser, it takes about 2-3 minutes to access, all the time.

I searched google, but found no useful answers to my problem. I know that I have to change the timeout setting for something, but not able to figure out which one. Can someone please help ?

Dave Newton · Accepted Answer · 2014-09-26T19:43:45.243

24

Use the :read_timeout option, specified in seconds, e.g.,

open('foo.com', :read_timeout => 10)

http://ruby-doc.org/stdlib-1.8.7/libdoc/open-uri/rdoc/OpenURI/OpenRead.html

edited Sep 26 '14 at 19:43

answered Sep 26 '14 at 19:26

Dave Newton

158,873
26
254
302

Thanks. How do I set the timeout to infinity ? Could not get that answer quickly from the link you gave. time out = -1 ??? – Erran Morad Sep 26 '14 at 19:32
I am getting the error - `syntax error, unexpected ':', expecting ')' response = open(url, read_timeout: 300)` – Erran Morad Sep 26 '14 at 19:33
@BoratSagdiyev ... You should probably use the correct map syntax for your Ruby version then. I'll edit my answer, but being able to extrapolate from code samples is a valuable skill. – Dave Newton Sep 26 '14 at 19:43
Thanks Dave. I just realized that I cannot use that argument because i have ruby 1.8. Is there an alternative for this ? Thanks. Error - `check_options': unrecognized option: read_timeout (ArgumentError)` – Erran Morad Sep 26 '14 at 19:50
@BoratSagdiyev Oy, I mis-read which version I was on when I was comparing versions--my fault! Will delete answer--sorry about that. – Dave Newton Sep 26 '14 at 19:52
No, please don'tdelete the answer. It is still useful as a reference. – Erran Morad Sep 26 '14 at 20:39
I ended up using ruby 2.x for this. Did not try 1.8 though. – Erran Morad Nov 18 '14 at 06:19

score 4 · Answer 2 · edited May 23 '17 at 12:00

The BufferedIO class that Net::HTTP uses for the connection that open-uri then uses for the request has a read_timeout attribute set to 60 seconds.

The Net::HTTP class provides a setter read_timeout for that.

Unfortunately, the way open-uri sets up that request doesn't provide you with a way to get at that setting before the request or the easily override the default of 60.

You will probably need to use Net::HTTP yourself.

link = URI.parse(url)
request = Net::HTTP::Get.new(link.path)
response = Net::HTTP.start(link.host, link.port) {|http|
  http.read_timeout = 100 #Default is 60 seconds
  http.request(request)
}

Code stolen from this answer

edit: or upgrade to 1.9 which supports the :read_timeout = x option Dave Newton noted.

Preventing timeout when connecting to a URL

2 Answers2

Linked