4

I'm using Ruby Net::HTTP under a NginX/Phusion Passenger server, attempting to POST a JSON string to the server. It appears that my POST, only when sending as 'application/json', prematurely closes the session to the server. I.E.:

On the server side:

127.0.0.1 - - [03/Sep/2013 07:47:14] "POST /path/to/submit " 200 45 0.0013
pid=12893 thr=47197563169088 file=ext/nginx/HelperAgent.cpp:933 time=2013-09-03 07:47:14.830 ]: Uncaught exception in PassengerServer client thread:
exception: Cannot read response from backend process: Connection reset by peer (104)
backtrace:
 in 'void Client::forwardResponse(Passenger::SessionPtr&, Passenger::FileDescriptor&, const Passenger::AnalyticsLogPtr&)' (HelperAgent.cpp:698)
 in 'void Client::handleRequest(Passenger::FileDescriptor&)' (HelperAgent.cpp:859)
 in 'void Client::threadMain()' (HelperAgent.cpp:952)

A client side debug session is:

opening connection to hostname.com...
opened
<- "POST /path/to/submit HTTP/1.1\r\nContent-Type: application/json\r\nAccept: application/json\r\nUser-Agent: agent+test\r\nConnection: close\r\nHost: hostname.com\r\nContent-Length: 660\r\n\r\n"
<- "[{json string}]"
-> "HTTP/1.1 301 Moved Permanently\r\n"
-> "Server: nginx/1.2.6\r\n"
-> "Date: Tue, 03 Sep 2013 14:47:15 GMT\r\n"
-> "Content-Type: text/html\r\n"
-> "Content-Length: 184\r\n"
-> "Connection: close\r\n"
-> "Location: https://hostname.com/path/to/submit\r\n"
-> "\r\n"
reading 184 bytes...
-> "<html>\r\n<head><title>301 Moved Permanently</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>301 Moved Permanently</h1></center>\r\n<hr><center>nginx/1.2.6</center>\r\n</body>\r\n</html>\r\n"
read 184 bytes
Conn close
opening connection to hostname.com...
opened
<- "POST /path/to/submit HTTP/1.1\r\nContent-Type: application/json\r\nAccept: application/json\r\nUser-Agent: agent+test\r\nConnection: close\r\nHost: hostname.com\r\nContent-Length: 660\r\n\r\n"
<- "[{json string}]"
-> "HTTP/1.1 200 OK\r\n"
-> "Content-Type: text/html;charset=utf-8\r\n"
-> "Content-Length: 45\r\n"
-> "Connection: close\r\n"
-> "Status: 200\r\n"
-> "X-Powered-By: Phusion Passenger (mod_rails/mod_rack) 3.0.17\r\n"
-> "X-Frame-Options: SAMEORIGIN\r\n"
-> "X-XSS-Protection: 1; mode=block\r\n"
-> "X-Content-Type-Options: nosniff\r\n"
-> "Server: nginx/1.2.6 + Phusion Passenger 3.0.17 (mod_rails/mod_rack)\r\n"
-> "Strict-Transport-Security: max-age=31536000\r\n"
-> "X-Frame-Options: DENY\r\n"
-> "\r\n"
reading 45 bytes...
-> "some url string"  <-- this is suspicous to me.
read 45 bytes
Conn close

The code I'm using to submit my data is:

private
def _http_client(method = 'get', location = nil, limit = 10) 
    raise ArgumentError, 'FATAL: too many HTTP redirects attempted...' if limit == 0
    response = nil 

    if location.nil?
        if @uri.nil?
            raise ArgumentError 'FATAL: no location specified. Quitting.'
        end 
    else
        @uri = URI.parse(location)
    end 

    client = Net::HTTP.new(@uri.host,@uri.port)
    client.open_timeout    = 15 # in seconds
    client.read_timeout    = 15 # in seconds
    initheader             = {'content-type' => 'application/json',
                              'accept'       => 'application/json',
                              'user-agent'   => UA                ,}
    client.set_debug_output $stderr  # XXX TEMPORARY

    if @uri.scheme == 'https'
        client.use_ssl = true
        client.verify_mode = 0 
    end 

    if method == 'post'
        serialized = JSON.generate(@payload)
        response   = client.send_request('POST',@uri.path,serialized,initheader = initheader)
        puts serialized
    end 

# Only intended for pings. In order to do
# full GETs (including query_params) this
# will need to be refactored a little.
    if method == 'get'
        response = client.get(@uri.path)
    end 

    case response
        when Net::HTTPSuccess
            @status = true
        when Net::HTTPMovedPermanently
            @status = false
            limit = limit - 1 
            _http_client(method,response['location'],limit)
        when Net::HTTPClientError
            $stderr.puts 'Client error.'
            @status = false
        when Net::HTTPServerError
            $stderr.puts 'Server error.'
            @status = false
        else
            $stderr.puts 'No idea what the server returned.'
            @status = false
    end 

    unless response.nil?
        @response  = {:code => response.code, :body => (response.body || 'OK')}
        @code      = @response[:code].to_s
        @http_body = @response[:body].to_s
    else
        $stderr.puts 'response from server was empty. :('
    end 

    _send_status
end 

I'm using:

  • Ruby version: 1.9.3p448
  • NginX 1.2.6
  • Passenger 3.0.17

I'm pretty sure I might be doing something wrong, but I'm new to Ruby and semi-new to this kind of web programming.

The code snippet above uses the send_request method but I've also tried post, and post2, resulting in the same outcome.

It's really suspicious to me that as soon as I use 'application/json' in my initheader everything breaks. When I remove it, everything works alright.

Jim
  • 1,499
  • 1
  • 24
  • 43
  • Have you consider move this to github? – fotanus Sep 03 '13 at 16:24
  • 1
    Personally, I'd use something besides Net::HTTP to do this unless absolutely backed into a corner. The [HTTPClient](https://github.com/nahi/httpclient), [RestClient](https://github.com/rest-client/rest-client) gems, [Typhoeus](https://github.com/typhoeus/typhoeus) or [Curb](https://github.com/taf2/curb) gems would save you a lot of work. – the Tin Man Sep 03 '13 at 17:00
  • I just started looking at the code. This is suspicious, but I haven't tried running the code to see if it's the problem: `response = client.send_request('POST',@uri.path,serialized,initheader = initheader)`. Why use `initheader = initheader`? Also, pay attention to proper white-spacing after commas. It's a readability thing. – the Tin Man Sep 03 '13 at 17:09
  • @the Tin Man Good callout (initheader = initheader). I have already tested removing the initheader LVAL. Same results, though, according to my understanding of the args in net/http.rb, initheader = initheader should be acceptable usage...*I think*. – Jim Sep 03 '13 at 17:24
  • @the Tin Man I'll see about using one of the other modules. I'm 95% backed into a corner here, but I will see about using a more friendly module. – Jim Sep 03 '13 at 17:25
  • `initheader = initheader` is benign, but it isn't "acceptable usage". It's assigning `initheader` to itself, and then, as a consequence of how Ruby works, will pass the value of `initheader` on to the method as a parameter. But, it's an error in waiting during maintenance, because it's unclear what you meant or why you'd write it that way. `=` is assignment, `==` is comparison. `==` would be more common, but then that would return a boolean and pass it on. Ruby v2 allows named parameters but that'd be `initheader: initheader`. – the Tin Man Sep 03 '13 at 17:44
  • @theTinMan good to know!! Thanks! Unfortunately, using just initheader in the call to send_request, still fails. I'm doing other testing as well, right now. I'm trying to determine if the problem is with the server or my call. I'm starting to lean toward the server being the problem. – Jim Sep 03 '13 at 17:48

1 Answers1

1

Try upgrading to Phusion Passenger 4. It handles I/O asynchrony better and in many cases avoids premature connection close problems.

Hongli
  • 18,682
  • 15
  • 79
  • 107
  • This seems to be the probable issue based from other readings I've found. Problem is, it seems Phusion isn't free anymore?? I've looked for about three minutes so I might be wrong. More to come. If it's not free anymore then I'm going to be stuck with v3. My company has no desire or need to buy a licensed copy since trying to articulate the need for it would be more time than it's worth, I'm afraid. – Jim Sep 05 '13 at 17:35
  • Ah, tricksters. Alright. Got it. Unfortunately, getting this bad boy to work in my environment is a lengthy process. It will take a couple of hours at least. I'll update when I get some data. Thanks Hongli. – Jim Sep 05 '13 at 17:43
  • Phusion Passenger has two variant, an open source (free) one and a commercial (paid) one. We also happen to be working on Debian packages, maybe in the future installation will be much easier for you. – Hongli Sep 05 '13 at 18:00