How can I tell if my Ruby server script is being overloaded?

Question

I have a daemonized ruby script running on my server that looks like this:

@server = TCPServer.open(61101)                         
loop do                                                 
  @thr = Thread.new(@server.accept) do |sock|
    Thread.current[:myArrayOfHashes] = []   # hashes containing attributes of myObject
    SystemTimer.timeout_after(5) do
      Thread.current[:string] = sock.gets
      sock.close

      # parse the string and load the data into myArrayOfHashes

      Myobject.transaction do           # Update the myObjects Table
        Thread.current[:myArrayOfHashes].each do |h|
          Thread.current[:newMyObject] = Myobject.new
          # load up the new object with data
          Thread.current[:newMyObject].save
        end
      end

    end
  end
  @thr.join
end

This server receives and manages data for my rails application which is all running on Mac OS 10.6. The clients call the server every 15 minutes on the 15 and while I currently only have 16 or so clients calling every 15 min on the 15, I'm wondering about the following:

1. If two clients call at close enough to the same time, will one client's connection attempt fail?
2. How I can figure out how many client connections my server can accommodate at the same time?
3. How can I monitor how much memory my server is using?

Also, is there an article you can point me toward that discusses the best way to implement this kind of a server? I mean can I have multiple instances of the server listening on the same port? Would that even help?

I am using Bluepill to monitor my server daemons.

Answer 1

1 and 2
The answer is no, two clients connecting close to each other will not make the connection fail (however multiple clients connecting may fail, see below).

The reason is the operating system has a default so called listening queue built into all server sockets. So even if you are not calling accept fast enough in your program, the OS will still keep buffering incoming connections for you. It will buffer these connections for as long as the listening queue does not get filled.

Now what is the size of this queue then?

In most cases the default size typically used is 5. The size is set after you create the socket and you call listen on this socket (see man page for listen here).

For Ruby TCPSocket automatically calls listen for you, and if you look at the C-source code for TCPSocket you will find that it indeed sets the size to 5:

https://github.com/ruby/ruby/blob/trunk/ext/socket/ipsocket.c#L108

SOMAXCONN is defined as 5 here:

https://github.com/ruby/ruby/blob/trunk/ext/socket/mkconstants.rb#L693

Now what happens if you don't call accept fast enough and the queue gets filled? The answer is found in the man page of listen:

The backlog argument defines the maximum length to which the queue of pending connections for sockfd may grow. If a connection request arrives when the queue is full, the client may receive an error with an indication of ECONNREFUSED or, if the underlying protocol supports retransmission, the request may be ignored so that a later reattempt at connection succeeds.

In your code however there is one problem which can make the queue fill up if more than 5 clients try to connect at the same time: you're calling @thr.join at the end of the loop.

What effectively happens when you do this is that your server will not accept any new incoming connections until all your stuff inside your accept-thread has finished executing.

So if the database stuff and the other things you are doing inside the accept-thread takes a long time, the listening queue may fill up in the meantime. It depends on how long your processing takes, and how many clients could potentially be connecting at the exact same time.

3
You didn't say which platform you are running on, but on linux/osx the easiest way is to just run top in your console. For more advanced memory monitoring options you might want to check these out:

ruby/ruby on rails memory leak detection
track application memory usage on heroku