1

I have created a client server system server by following the CppRestSdk examples.

Here is the sample code section of my implementation (only the request sending section).

nf.request(methods::POST, U("/"), requestBody)
    .then([=] (pplx::task<http_response> task)
    {
        LOG_ENTRY;
        // some other work ..
        resp = task.get(); // fails
        http_response resp;
        do {
            try {

                resp = task.get();
            } catch (std::exception& e) {
                higLog("EXCEPTION CAUGHT: %s", e.what());

            }

            // some other checks ..

        }while(0);

       // process resp later ..

    }

where nf is a http_client.

Here is the architecture of my system.

architecture

thread_pool_call_flow

The main thread runs an epoll loop in the front-end server. It also creates an explicit thread-pool. The epoll loop receives the incoming messages from the client side and stores those messages in a work_queue (implemented by me). One of the threads (pool size of 500) in the thread pool wakes up and picks a job from the work queue. This thread performs processing on the message and talks to back-end server using cpprestsdk http_client object.

For a small number (10 or 15) of concurrent threads in the client side, the system works fine. But when I increase the number of threads (50 threads for example) int the client side, the front-end server starts showings unpredictable errors. Mostly it blocks for a long time in the task.get() function. Sometimes it also shows "Failed to read HTTP status line" int he catch block.

I am new to cpprestsdk. I am not sure what is happening really. But to my guess backlog queue size is small in the back-end server. I found a set_backlog() function, but I can not use it because of const reference. line 307.

What could be the reason for the above errors? It is common behavior? Is it because of 40 threads inside cpprestsdk thread pool ?

Debashish
  • 1,155
  • 19
  • 34
  • That reads like a scaling issue of the server. How many concurrent requests your server can handle i.e. `epoll`? And, how are they being processed? Added latency during processing? Did you do some kind of benchmark or load testing use a tool e.g. `Apache Bench`, SoapUI, etc.? – Azeem Mar 19 '19 at 06:07
  • No benchmark. I am using my own load-test program on the client side. The server can handle 10 threads with the server using `thread pool`. Without `thread-pool` server can handle `100+` client threads(i.e. server is having only one main threads + `40 cpprestsdk` threads). – Debashish Mar 19 '19 at 07:21
  • Are you sure the sdk is thread safe? Checked the whole documentation, FAQ and their issue tracker? Gone by the open issue count, this library real seems to have problems. – Superlokkus Mar 19 '19 at 08:53

0 Answers0