4

We have a streaming server implemented using live555 library. This server is deployed on CentOS instance.

Recently we wanted to modify the server socket options, so that it can accept the requests immediately after re-starting the process (either after crash or manual restart).

I have referred to man pages and few web links and set the socket options (SO_REUSEADDR and SO_REUSEPORT) before bind() call.

int setupStreamSocket(UsageEnvironment& env,
                      Port port, Boolean makeNonBlocking) {


  int newSocket = createSocket(SOCK_STREAM);
  if (newSocket < 0) {
    socketErr(env, "unable to create stream socket: ");
    return newSocket;
  }

  int reuseFlag = 1;
  fprintf(stderr,"reuseFlag : %d\n",reuseFlag);

  if (setsockopt(newSocket, SOL_SOCKET, SO_REUSEADDR,
                 (const char*)&reuseFlag, sizeof reuseFlag) < 0) {
    socketErr(env, "setsockopt(SO_REUSEADDR) error: ");
    closeSocket(newSocket);
    return -1;
  }

  if (setsockopt(newSocket, SOL_SOCKET, SO_REUSEPORT,
                 (const char*)&reuseFlag, sizeof reuseFlag) < 0) {
    socketErr(env, "setsockopt(SO_REUSEPORT) error: ");
    closeSocket(newSocket);
    return -1;
  }
  int flag = 1;
  setsockopt( newSocket, IPPROTO_TCP, TCP_NODELAY, &flag, sizeof(flag) );

  if (port.num() != 0 || ReceivingInterfaceAddr != INADDR_ANY) {

    MAKE_SOCKADDR_IN(name, ReceivingInterfaceAddr, port.num());
    int reuse_addr_val, reuse_port_val;
    socklen_t reuse_addr_len = sizeof(reuse_addr_val);
    socklen_t reuse_port_len = sizeof(reuse_port_val);
    getsockopt(newSocket, SOL_SOCKET, SO_REUSEADDR, &reuse_addr_val, &reuse_addr_len);
    getsockopt(newSocket, SOL_SOCKET, SO_REUSEPORT, &reuse_port_val, &reuse_port_len);
    fprintf(stderr,"reuse_addr_val = %d, reuse_port_val = %d\n", reuse_addr_val, reuse_port_val);

    if (bind(newSocket, (struct sockaddr*)&name, sizeof name) != 0) {
      char tmpBuffer[100];
      sprintf(tmpBuffer, "bind() error (port number: %d): ",
              ntohs(port.num()));
      socketErr(env, tmpBuffer);
      closeSocket(newSocket);
      return -1;
    }
  }

  if (makeNonBlocking) {
    if (!makeSocketNonBlocking(newSocket)) {
      socketErr(env, "failed to make non-blocking: ");
      closeSocket(newSocket);
      return -1;
    }
  }

  return newSocket;
}

This code works as expected (binding to the address even when a socket is in TIME_WAIT state) if I restart the server with above options.

If I replace previous build (without socket options) with the build created with above code then I have noticed that occasionally bind() is failing with Address already in use error.

When the bind() failed the port/address is in TIME_WAIT state. So the server should able to bind the socket to the address.

tcp 0 0 10.0.1.24:8554 10.0.1.89:27085 TIME_WAIT -

And the getsockopt() of my code printed the flag value (corresponding to SO_REUSEADDR and SO_REUSEPORT) as 1.

reuse_addr_val = 1, reuse_port_val = 1

bind() error (port number: 8554): Address already in use

So I'm wondering why it's failing only few times. Did I miss something in my code ? or is this an expected behavior ?

Community
  • 1
  • 1
Durgesh Tanuku
  • 1,124
  • 2
  • 10
  • 26
  • `REUSE_PORT` only works, if the first process binding this port also has this option set, maybe this is an issue here? – Ctx Aug 18 '18 at 12:59
  • Re-reading your post, it seems like you do not need `REUSE_PORT` at all, if you have only a single process binding the port at a time. – Ctx Aug 18 '18 at 13:04
  • Initially I have only used SO_REUSEADDR as socket option. I have not used SO_REUSEPORT. The behavior was the same. – Durgesh Tanuku Aug 18 '18 at 13:16
  • Hm, at first glance I would say, there _has_ to be another process which has that port bound if the `bind()` fails with EADDRINUSE. You write "the port/address is in TIME_WAIT state". Indeed, only a connection can be in that state. Did you make sure, that there isn't a second line in netstat with a LISTEN state for port 8554? – Ctx Aug 18 '18 at 13:20
  • Yes, I have checked 2 to 3 times. There was no other socket associated with 8554 in LISTEN or any other state. FYI, I have used "netstat -tnap | grep 8554" – Durgesh Tanuku Aug 18 '18 at 13:25

0 Answers0