0

I am trying to evaluate the performance of libuv because of the promise that it can manage 100,000 of TCP sockets and can generate 10,000 of new TCP sessions/second.

Created the following code snippet based on a gist to validate the performance of libuv. However, when it runs against a server, it's surprisingly slow according to wireshark pcap capture

  • TCP handshakes were done one after another, I was expecting TCP SYNs to be sent in parallel
  • The TCP data was sent after all the sessions were established. I had expected the TCP data to be sent as soon as the TCP handshake is complete.

This test was run on Ubuntu 14.04 (64bit core i7 cpu).

Not sure if there is problem with libuv or with this code snippet.

I know this has some memory leaks, but it doesn't matter since I am only doing evaluation.

#include <stdio.h>
#include <stdlib.h>
#include <uv.h>

//based on https://gist.githubusercontent.com/snatchev/5255976/
//raw/8392c42d719bb775053036e32b21affdf932c1b7/libuv-tcp-client.c
static void on_close(uv_handle_t* handle);
static void on_connect(uv_connect_t* req, int status);
static void on_write(uv_write_t* req, int status);

static uv_loop_t *loop;

static uv_buf_t alloc_cb(uv_handle_t* handle, size_t size) {
  return uv_buf_init(malloc(size), size);
}

void on_close(uv_handle_t* handle)
{
  printf("closed.");
}

void on_write(uv_write_t* req, int status)
{
  if (status) {
    uv_err_t err = uv_last_error(loop);
    fprintf(stderr, "uv_write error: %s\n", uv_strerror(err));
        return;
  }
    printf("wrote.\n");
    free(req);
    //uv_close((uv_handle_t*)req->handle, on_close);
}

void on_read(uv_stream_t* tcp, ssize_t nread, uv_buf_t buf)
{
    printf("on_read. %p\n",tcp);
    if(nread >= 0) {
        //printf("read: %s\n", tcp->data);
        printf("read: %s\n", buf.base);
    }
    else {
        //we got an EOF
    uv_close((uv_handle_t*)tcp, on_close);
    }

    //cargo-culted
    free(buf.base);
}

void write2(uv_stream_t* stream, char *data, int len2) {
    uv_buf_t buffer[] = {
        {.base = data, .len = len2}
    };
    uv_write_t *req = malloc(sizeof(uv_write_t));
    uv_write(req, stream, buffer, 1, on_write);
}

void on_connect(uv_connect_t* connection, int status)
{
    if (status < 0) {
        printf("failed to connect\n"); return;
    }
    printf("connected. %p %d\n",connection, status);

    uv_stream_t* stream = connection->handle;
    free(connection);
    write2(stream, "echo  world!", 12);
    uv_read_start(stream, alloc_cb, on_read);
}

void startConn(char *host, int port) {
    uv_tcp_t *pSock = malloc(sizeof(uv_tcp_t));
    uv_tcp_init(loop, pSock);
    uv_tcp_keepalive(pSock, 1, 60);

    struct sockaddr_in dest = uv_ip4_addr(host, port);

    uv_connect_t *pConn = malloc(sizeof(uv_connect_t));
    printf("allocated %p\n", pConn);
    uv_tcp_connect(pConn, pSock, dest, on_connect);
}

int main(int argc, char **argv) {
    loop = uv_default_loop();
    int i;
    for (i=0; i<10; i++)
        startConn("0.0.0.0", 1234);

    uv_run(loop, UV_RUN_DEFAULT);
}
packetie
  • 4,839
  • 8
  • 37
  • 72
  • 1. did you try this without the `printf` (they induce a level of syncing due to blocking IO access)? ... 2. I wonder, does this happen also when `libuv` acts as a server? – Myst May 01 '17 at 02:31
  • Thanks @Myst for the suggestion, commented out the `printf` didn't make any difference. I am interesting on libuv as client right now, will try the server side in the future. – packetie May 01 '17 at 03:18
  • Another thought, I know it's a less stable API, but did you try using `uv_stream_set_blocking(pSock, 0)` before calling `uv_tcp_connect`? – Myst May 01 '17 at 03:28
  • I got error `gcc delme.c -luv /tmp/ccDEruuO.o: In function `startConn': delme.c:(.text+0x282): undefined reference to `uv_stream_set_blocking' collect2: error: ld returned 1 exit status`, don't know why. – packetie May 01 '17 at 03:43
  • Turned out I was using an older version of libuv (that one that comes with ubuntu 14.04). Built libuv1.0 from source and I was able to resolve symbol `uv_stream_set_blocking`. Unfortunately I still get the same issue where TCP handshakes happen sequentially, not in parallel as I hoped. – packetie May 01 '17 at 04:11
  • Do you get the same issue with other libraries (i.e. `libevent` or [`facil.io`](https://github.com/boazsegev/facil.io/blob/ae77d2e1fce30b53f1b21c6cce1a18d25c7e7008/tests/client.h))... I'm biased regarding `facil.io`, but I'm also wondering if it's a testing issue or a library issue. – Myst May 01 '17 at 05:39
  • No need to use uv_stream_set_blocking, it's already set to zero internally. What are you evaluating, the server or the client part? Being single-threaded, no i/o can happen in parallel, you can, however, use multiple loops in multiple threads to fully utilize your resources. – saghul May 01 '17 at 06:26
  • have you run a profiler on the code, A profiler will tell you where the code is spending its' time (probably blocked I/O since the 'parallel/asynchronous' capabilities have not been enabled.) – user3629249 May 01 '17 at 21:43
  • Isn't it more appropriate to test for the TCP ACK than the TCP SYN? SYNs are sent immediately and the `connect` function returns, non-blocking until a connection was established (`ACK`). Once a connection was established, the event is fired. When tested against two servers with different speeds (i.e., a remote vs. local server or a slow vs. fast), the ACKs should be in a different order than the SYN signals. – Myst May 02 '17 at 19:06
  • `facil.io` looks like a nice framework, very flexible but it's not much faster than the dumb server I have tested again. I suspect that `connect` didn't immediately return because all the SYN packets were sent after previous handshakes. Wish there is an easy way to check if `parallel/asynchronous` capabilities have been enabled as in previous comment. – packetie May 06 '17 at 14:03

1 Answers1

0

This example is compiled against libuv 0.1.

After it's adapted to compile and run against libuv1.x, it works as expected. Here is the gist.

There is considerable difference between libuv 0.1 and libuv 1.0. Hope this version could be helpful to those who are looking for a working example of libuv.c. By the way, there are many good examples in libuv repo in github, check the "test" subdirectory.

packetie
  • 4,839
  • 8
  • 37
  • 72