Net::HTTP Proxy list

Question

I understand that you could use proxy in the ruby Net::HTTP. However, I have no idea how to do this with a bunch of proxy. I need the Net::HTTP to change to another proxy and send another post request after every post request. Also, is it possible to make the Net::HTTP to change to another proxy if the previous proxy is not working? If so, how? Code I'm trying to implement the script in:

require 'net/http'
sleep(8)
http = Net::HTTP.new('URLHERE', 80)
http.read_timeout = 5000
http.use_ssl = false
path = 'PATHHERE'
data = '(DATAHERE)'
headers = {
'Referer' => 'REFERER HERE',
'Content-Type' => 'application/x-www-form-urlencoded; charset=UTF-8',
'User-Agent' => '(USERAGENTHERE)'}

resp, data = http.post(path, data, headers)


# Output on the screen -> we should get either a 302 redirect (after a successful login) or an error page
puts 'Code = ' + resp.code
puts 'Message = ' + resp.message
resp.each {|key, val| puts key + ' = ' + val}
puts data

end

Have you been able to get it working with just one proxy? If so, would you post the code? — Travis Hohl, Oct 06 '14 at 15:39

Travis Hohl · Answer 1 · 2014-10-24T12:46:27.370

Given an array of proxies, the following example will make a request through each proxy in the array until it receives a "302 Found" response. (This isn't actually a working example because Google doesn't accept POST requests, but it should work if you insert your own destination and working proxies.)

require 'net/http'

destination = URI.parse "http://www.google.com/search"
proxies = [
    "http://proxy-example-1.net:8080",
    "http://proxy-example-2.net:8080",
    "http://proxy-example-3.net:8080"
]

# Create your POST request_object once
request_object = Net::HTTP::Post.new(destination.request_uri)
request_object.set_form_data({"q" => "stack overflow"})

proxies.each do |raw_proxy|

    proxy = URI.parse raw_proxy

    # Create a new http_object for each new proxy
    http_object = Net::HTTP.new(destination.host, destination.port, proxy.host, proxy.port)

    # Make the request
    response = http_object.request(request_object)

    # If we get a 302, report it and break
    if response.code == "302"
        puts "#{proxy.host}:#{proxy.port} responded with #{response.code} #{response.message}"
        break
    end
end

You should also probably do some error checking with begin ... rescue ... end each time you make a request. If you don't do any error checking and a proxy is down, control will never reach the line that checks for response.code == "302" -- the program will just fail with some type of connection timeout error.

See the Net::HTTPHeader docs for other methods that can be used to customize the Net::HTTP::Post object.

Net::HTTP Proxy list

1 Answers1