4

I am trying to understand the core principles of non-blocking programming (and frameworks like project reactor). The main idea is to have "thread pool" with determined number of threads (executors) and tasks which are executed there. We should not have any blocked threads. In "user code" we just run something to execute and leave callback (what to do with the result). Out "user" thread is not blocked, right. But what if my task depends on some jdbc query. My task will request this query and then will be blocked waiting for the result, right? So, this thread is blocked.

But we avoid thread creating (which is expensive). Is it the core benefit of this style?

If my thread pool consists of 2 executors and both are blocked waiting for something, other tasks will not be executed, right? How to avoid it? Create more than 2 threads?

Вадим Парафенюк
  • 579
  • 2
  • 7
  • 14
  • 3
    don't use JDBC, because it's a blocking API. That's why Spring is working on R2DBC, a non-blocking API, using reactive, non-blocking database drivers. The goal is not to improve performance, it's to improve scalability: being able to process a lot of concurrent requests without needing a dedicated thread per request. – JB Nizet Nov 22 '18 at 11:42
  • Okey. Lets switch to Reactive Mongo Driver. It's non-blocking. But when I query something it has to wait for the result somewhere, right? I mean... My query is a set of TCP packets and some thread MUST wait for response TCP packets, right? And while it's waiting it's blocked anyway. Correct me if I'm wrong please :) It's a key to understand this style – Вадим Парафенюк Nov 22 '18 at 11:47
  • https://stackoverflow.com/a/41227444/6162023 Non-Blocking IO (or Asynchronous IO) tells the relevant driver (the kernel, a DB driver, etc.) to initialize an IO action and then the thread keeps on doing other stuff. depending on the technology you use, you handle asynchronous IO results (which may be even an exception) in callbacks (such in Node.js) , channels (Java) , futures (C++) , promises (newer versions of Node.js), Tasks (.Net), coroutines(C++17) etc. – Sofo Gial Nov 22 '18 at 11:49
  • 1
    Thank you a lot for the very useful link :) "Asynchronous IO does not use threads to make the IO asynchronous. this is the key point here". It's just what I needed to know :) – Вадим Парафенюк Nov 22 '18 at 11:57

2 Answers2

16

Threads are relatively costly system resources. For example, each thread needs memory for the call stack. How much this is depends on the operating system, but typically it's something like 1 or 2 MB. This means it's not a good idea to start thousands of threads - you'd waste 1 or 2 GB memory just on the call stacks of 1000 threads.

So, to do things more efficiently you want to limit the number of threads, for example using a thread pool to handle work. The thread pool makes it possible to manage the number of threads that are being used.

However, imagine that you'd have a thread pool with 10 threads, and then 10 requests come in. Each of your threads will be reserved to handle a request. While they are busy, you can't handle request #11 because there is no thread free. When you are using blocking I/O, then, even though all your 10 threads are doing nothing (waiting for I/O to complete), request #11 cannot be handled...

When you use non-blocking I/O, threads will never need to wait for I/O - so when the handling request #3 is suspended because it needs the result of an I/O operation, the thread that was handling it can temporarily switch to handling other requests.

So, with non-blocking I/O, you never have waiting threads and you are using system resources more efficiently.

This will only work if you are using non-blocking I/O from the front to the back of your system. If at the back-end you are using JDBC, which is a blocking API, then you'll loose the full benefit of non-blocking I/O.

Therefore, if you have a database at the back-end, this works best if you have a DB which supports non-blocking I/O. Some NoSQL databases like MongoDB support this, and for some relational databases there are special drivers / APIs available that support this. You won't be using JDBC in that case, because JDBC is an inherently blocking API.

Oracle is working on a new API for relational databases tentatively called ADBA which will allow you to do non-blocking / async I/O with relational databases but it's not ready yet.

Jesper
  • 202,709
  • 46
  • 318
  • 350
  • Thanks for the rich explanation. When we receive a request and it goes to my non blocking controller (that returns for example Mono) in some thread pool like reactor http thread pool and this controller returns a Mono. So, receive task is done here (when return Mono) and new task to response this request will be created later (when mongodb returns needed object), right? – Вадим Парафенюк Nov 22 '18 at 13:06
  • Here is the great example how your application uses threads https://kamilszymanski.github.io/resources-utilization-in-reactive-services/ – kojot Nov 22 '18 at 13:32
  • @ВадимПарафенюк Yes, when you use for example Spring WebFlux for reactive programming then the reactive library will handle the threading for you, and the actual work is not done at the moment your controller method is called, but at the moment when the actual data is returned by the DB. – Jesper Nov 22 '18 at 13:38
  • @Jesper So web framework (Spring WebFlux in this case) at first gets my Publisher (that will publish data from DB + pipelines) from required controller and subscribes it to some magic subscriber that can collect elements (if it Flux) and then send it as response (after onComplete). Is it rough description of WebFlux workflow? – Вадим Парафенюк Nov 22 '18 at 15:19
  • @ВадимПарафенюк Yes, that's roughly how it works; the framework gets a publisher from your controller (a Mono or Flux) and then subscribes to it to get the actual work done, in such a way that no threads have to block. – Jesper Nov 22 '18 at 19:51
2

Project Reactor is an implementation of Reactive Streams specification. The specification overview can be found at ReactiveManifest. It's not just creating a set of threads and letting them do their jobs, It's the framework or the runtime (in this case ProjectReactor) that will organize your code in such a way that it'll presumably behave as nonblocking. Also, the whole system implementation has to be in this fashion otherwise you won't be benefited from the reactive streams.

If my thread pool consists of 2 executors and both are blocked waiting for something, other tasks will not be executed, right? How to avoid it? Create more than 2 threads?

The answer to this will be yes, and no. The framework may are may not create threads. Since the code will be interleaved among the threads, Since the non-blocking system are event-driven including the low-level operations (ex, libuv I/O), It's not necessary for a thread to wait for the completion of an I/O operation. Meanwhile, the thread will be executing something meaningful. The completion of the task will be notified and the dependent code can be executed by any of the available thread. The goal of such a system is to utilize the CPU to the fullest with limited resources(threads).

Taken from http://www.reactive-streams.org. The main goal of Reactive Streams is to govern the exchange of stream data across an asynchronous boundary—think passing elements on to another thread or thread-pool—while ensuring that the receiving side is not forced to buffer arbitrary amounts of data. In other words, back pressure is an integral part of this model in order to allow the queues which mediate between threads to be bounded. The benefits of asynchronous processing would be negated if the communication of back pressure were synchronous (see also the Reactive Manifesto), therefore care has to be taken to mandate fully non-blocking and asynchronous behavior of all aspects of a Reactive Streams implementation.

It's the Reactor framework that enforces and help you in building a completely non-blocking system from the ground up.

imvinaypatil
  • 31
  • 4