How to re bind a udp socket in Linux

Question

I am an experienced Linux socket programmer and am writing a server application which has many outgoing interfaces. Now server socket binds to a random source port in the start of the process along with INADDR_ANY.

Later at some point when submitting response to a specific node, i need to assign a fixed source ip address. The standard way to do this is calling bind. However, bind is called once for the port number, successive calls fail with invalid argument error.

Creating a new socket is not really a good choice since i will have to be doing this very often upon responding to some clients.

I have also explored SO and a lot of socket options such as IP_FREEBIND, but it doesn't quite suite my scenario.

Perhaps using IP_PKT_INFO and setting source address might work unless it suffers the same problem i.e. not allowing a socket once bound to INADDRANY to rebind to a fixed source ip latter.

Is there a way to unbind an existing socket or an alternate way to setting source ip address in outgoing packet?

    int sock = socket(AF_INET, SOCK_DGRAM, 0);

    if(sock < 0)
        printf("Failed creating socket\n");

    struct sockaddr_in addr;
    memset(&addr, 0, sizeof(struct sockaddr_in));
    addr.sin_family = AF_INET;
    addr.sin_port = htons(1500);
    addr.sin_addr.s_addr = INADDR_ANY;

    // first bind succeeds
    if ( (status = bind(sock, (struct sockaddr *) &addr, sizeof(addr))) < 0)
        printf("bind error with port %s\n", strerror(errno));  

    struct sockaddr_in src_addr;
    memset(&src_addr, 0, sizeof(struct sockaddr_in));
    src_addr.sin_family = AF_INET;
    if (inet_aton("10.0.2.17", &(src_addr.sin_addr)) == 0)
        printf("Failed copying address\n");

    // second bind fails
    if((status = bind(sock, (struct sockaddr *)&src_addr, sizeof(src_addr))) < 0)
        printf("re bind error with ip %s\n", strerror(errno));

Any ideas in this regard will be highly appreciated. I have gone through considerable material on sockets, SO etc. but no success yet.

Answer 1

I finally found the solution myself so accepting my own answer (shameless but correct plugin), supplemented with code sample.

I originally wanted to rewrite source address of an outgoing packet without creating the socket again where the socket was already bound. Calling bind multiple times fail for this case, and (in my particular situation), i was not able to just have separate sockets for each source ip and use it.

I found some references in IP_PACKET_INFO but it was a pain to get it to work correctly. Following reference was helpful.

Setting source of udp socket

Sample Code

Here is a trivial application which creates a udp socket, binds it to a local port, then before sending a particular message, it appends the outgoing source ip address. Keeping in mind that in my case, i created a sudo interface and assigned it another ip. The send call will fail if this is not the case.

int status=-1;
int sock = socket(AF_INET, SOCK_DGRAM, 0);

if(sock < 0)
    printf("Failed creating socket\n");

int opt = 1;
setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt));

struct sockaddr_in bind_addr;
memset(&bind_addr, 0, sizeof(struct sockaddr_in));
bind_addr.sin_family = AF_INET;
bind_addr.sin_port = htons(44000); // locally bound port

if((status = bind(sock, (struct sockaddr *)&bind_addr, sizeof(bind_addr))) < 0)
    printf("bind error with port %s\n", strerror(errno));

// currently using addr as destination
struct sockaddr_in addr;
memset(&addr, 0, sizeof(struct sockaddr_in));
addr.sin_family = AF_INET;
addr.sin_port = htons(80); // destination port
if (inet_aton("74.125.236.35", &(addr.sin_addr)) == 0)
    printf("Failed copying remote address\n");
else
    printf("Success copying remote address\n");

struct sockaddr_in src_addr;
memset(&src_addr, 0, sizeof(struct sockaddr_in));
src_addr.sin_family = AF_INET;
if (inet_aton("10.0.2.17", &(src_addr.sin_addr)) == 0)
    printf("Failed copying src address\n");
else
    printf("Success copying src address\n");

char cmbuf[CMSG_SPACE(sizeof(struct in_pktinfo))];

char msg[10] = "hello";
int len = strlen(msg);

struct msghdr mh;
memset(&mh, 0, sizeof(mh));

struct cmsghdr *cmsg;
struct in_pktinfo *pktinfo;

struct iovec iov[1];
iov[0].iov_base = msg;
iov[0].iov_len = len;

mh.msg_name = &addr; // destination address of packet
mh.msg_namelen = sizeof(addr);
mh.msg_control = cmbuf;
mh.msg_controllen = sizeof(cmbuf);
mh.msg_flags = 0;
mh.msg_iov = iov;
mh.msg_iovlen = 1;

// after initializing msghdr & control data to 
// CMSG_SPACE(sizeof(struct in_pktinfo))
cmsg = CMSG_FIRSTHDR(&mh);
cmsg->cmsg_level = IPPROTO_IP;
cmsg->cmsg_type = IP_PKTINFO;
cmsg->cmsg_len = CMSG_LEN(sizeof(struct in_pktinfo));
pktinfo = (struct in_pktinfo*) CMSG_DATA(cmsg);

//src_interface_index 0 allows choosing interface of the source ip specified
pktinfo->ipi_ifindex = 0;
pktinfo->ipi_spec_dst = src_addr.sin_addr;

int rc = sendmsg(sock, &mh, 0);
printf("Result %d\n", rc);

The key statement is

pktinfo->ipi_spec_dst = src_addr.sin_addr;

where we are specifying the source ip address to be used. The rest of things like cmsg struct etc. are merely used in order to be able to write ipoktinfo struct ourselves

Answer 2

There is no way to unbind and rebind an existing socket.

Answer 3

Why don't you create a socket for each interface instead? Since the UDP/IP protocol is connectionless, you can choose the source IP address by choosing which socket you use to send the reply with; there is no need to use the same socket the incoming datagram was received on.

The downsides are that you can no longer bind to the wildcard address, and you must use select(), poll(), multiple threads, or some other mechanism to receive datagrams from multiple sources concurrently. You'll also need some logic to efficiently pick the socket based on the client IP address.

In most cases, I suspect that adding a few route entries to route each remote IP address to the desired host IP address, and using a separate socket for each host IP address and port combination, solves the issues perfectly -- and using the very efficient kernel functionality to do so. While the behaviour may be an application requirement, I suspect it is better solved using the network interface configuration instead. Unfortunately, often the requirements are written by semi-functional idiots better suited for manual labor, and your hands are tied.. if so, I commiserate.

If you have a test network with workstations having multiple physical network interfaces, I can provide a simple example C99 test program you can use to verify the design works.