2

This question may be slightly off topic but I didn't know where else to ask. I was reading this https://github.com/msgpack-rpc/msgpack-rpc/blob/master/spec.md and saw that the specification included being able to send out of order messages using the same connection.

The only way I have done TCP socket programming before is by sending requests synchronously on a socket, for example, I will open a socket to 127.0.0.1, send a request to that server through that socket and wait for a response. When I get the response for the request I have sent I close that connection by calling close() on the client and close() on the server after I have responded to the request.

For background, I am working on a project in C++ with libevent to do something very similar to what RPC systems do, so I wanted to know what request response socket cycle I should use in the underlying transport system.

In C++ thrift there is a client method called open() that (presumably) opens a connection and keeps it open until you call close(). How does this work in systems where that is abstracted away? For example in that messagepack-RPC link I have included above. What is the best course of action? Open a connection if there is none, send the request and when all previous requests are served close that connection (on the server, call close() when all pending requests have been responded to)? Or do we have to somehow try and keep that connection alive for a period of time that extends beyond the request lifetimes? How will the server and the client know what that time period is? For example should we register a read event handler on the socket and close the connection when recv() returns 0?

If this is a problem that different systems solve differently then can someone direct me to a resource that I can use to read up on possible patterns for keeping connections alive (preferably in event driven systems)? For example I read that HTTP servers always keep the connection open, why is that? Doesn't keeping every single connection open mean that the server will essentially leak memory?

Sorry about the seemingly basic question, I have only ever done really simple TCP socket programming before, so I might not know how things are done.

Curious
  • 20,870
  • 8
  • 61
  • 146
  • The answer to this question is going to depend almost entirely on what you are trying to do, and what protocol is being used between your code and the code at the other end of the TCP connection. Some data protocols support persistent connections, others do not. That said, since there is a certain amount of overhead involved in setting up and tearing down a TCP connection, there is often some benefit it keeping one open and reusing it for future requests. – Jeremy Friesner Jun 07 '17 at 03:36
  • @JeremyFriesner and usually how do applications decide on how to do that? I feel like sending heartbeats might be slightly overkill but I don't know how to judge these decisions.. – Curious Jun 07 '17 at 05:25
  • My own applications just hold the socket open indefinitely (or until the user explicitly specifies he wants to disconnect), and include a four-byte word at the start of each data-message indicating the length of the message in bytes, so that the receiver knows how many bytes it should read before trying to parsing the message. Once that's implemented and working, you can periodically send small/dummy messages as heartbeats, but it's not really necessary unless you are trying to actively monitor the health of the connection. – Jeremy Friesner Jun 07 '17 at 14:29
  • @JeremyFriesner and is it a good idea to send a message to the server on a `close()` call by the user, for example in the thrift client I talked about. So the server knows it can call `close()` on the socket? – Curious Jun 07 '17 at 17:39
  • @JeremyFriesner I do not know if doing something like this would be idiomatic. Since essentially I would be dispatching I/O from the destructor of a class. – Curious Jun 07 '17 at 18:18
  • 4
    Don't bother sending a message, just call close(). The server can't rely on you sending an explicit "I'm-going-to-go-away-now" message anyway, because in cases where there is a network outage (or your client program crashes, or the user abruptly pulls the plug on the client machine), you won't be able to send one. So there's no point in sending a message when the server can (and must!) just see that the connection was closed based on recv()'s return value anyway. – Jeremy Friesner Jun 07 '17 at 19:14
  • @JeremyFriesner how does the server close the connection then? a timeout? and what happens if the server has closed the connection and the client sends another request on the same socket (which it thinks is currently open)? – Curious Jun 07 '17 at 19:16
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/146107/discussion-between-jeremy-friesner-and-curious). – Jeremy Friesner Jun 07 '17 at 19:16
  • @Curious When one end of a TCP session closes it, then the other end is (typically) notified that there is data to read and when they read they will read 0 bytes, which indicates the other end cleanly closed their end of the session. If a client tries to send on a session just after the other end closes it, it depends on the timing, but it might seem to send without issue or you might get a SIGPIPE / EPIPE error on the send telling the sender that the other end has disconnected. Typically you want to install a signal handler (or SIG_IGN) for SIGPIPE in these kinds of programs. – jschultz410 Jun 21 '17 at 03:02
  • Higher level, unless you have a good reason for synchronous, one-by-one calls, you want to allow pipelining of both requests and responses to achieve decent throughput over possibly high latency connections. – jschultz410 Jun 21 '17 at 03:05

1 Answers1

5
  1. Use a connection pool at the client.
  2. Have a background thread in the client that expires idle connections after some timeout.
  3. Write the server so it can handle multiple requests on each accepted socket, i.e. loop reading requests and sending responses until the peer closes the connection, or a read timeout occurs reading the request, or a socket error occurs.
  4. Don't send a close message from either side. Closng the socket is sufficient.
  5. Don't use an application ping message to keep the connection alive. If the peer or an intermediate router considers connections so expensive as to terminate them after some period, you don't have any business trying to fool it.

All this is much how most HTTP implementations already work. Note the timeouts at both ends to control each end's idle resource usage.

user207421
  • 305,947
  • 44
  • 307
  • 483
  • So even HTTP servers close their sockets on a timeout? Or do they wait until they can send a close header to the client? – Curious Jun 18 '17 at 14:24
  • Also you said loop reading requests until a read timeout occurs. What if the client sends a complete request to the server and that server responds and then that connection is idle for the remaining future? Should the timeout also cause that connection to be closed in that case? – Curious Jun 18 '17 at 14:26
  • 1. There is no 'close header' in HTTP. That's why I said 'this is much how most HTTP implementations work'. 2. The server will timeout reading the non-existent second request, as I said. No special handling is required for this case. The server should timeout all requests. – user207421 Jun 19 '17 at 11:03
  • @Curious: A server will very often keep a connection open as long as it has resources for it. If it starts to use too much memory, file descriptors, etc., then it can have a policy for how it reclaims those resources (e.g. - least recently used idle connections get closed and reclaimed). It can also start quickly "refusing" new connections if requests are coming too fast and too furious. – jschultz410 Jun 21 '17 at 03:12
  • @Curious TCP has this interesting feature that, by default and design, there are no closing timeouts built-in to the base protocol. By default, a TCP connection can remain open "forever," even if no data is being sent **AT ALL** between the two ends for arbitrarily long periods of time. This allows for things like connecting to a server at work, closing your laptop for a few hours, then reopening it and that same TCP session will still work (so long as your IP doesn't change). – jschultz410 Jun 21 '17 at 03:17
  • You can add a probe + close timeout to a TCP connection through SO_KEEP_ALIVE option on a socket, which will probe idle connections after some usually long (e.g. - 2 hours) timeout. If the other end doesn't respond, then an error will be provoked on the local end causing you to close it. Unfortunately, the keep alive timeout attached to a socket is usually fixed for the entire host and can't be varied on a per socket basis. – jschultz410 Jun 21 '17 at 03:19