3

I am trying to write a HTTP API server which does basic CRUD operation on a specific resource. It talks to an external db server to do the operations. Future support in scala is pretty good, and for all non-blocking computation, future is used. I have used future in many places where we wrap an operation with future and move on, when the value is eventually available and the call back is triggered.

Coming to an HTTP API server's context, it is possible to implement non-blocking asynchronous calls, but when a GET or a POST call still blocks the main thread right?

  1. When a GET request is made, a success 200 means the data is written to the db successfully and not lost. Until the data is written to the server, the thread that was created is still blocking until the final acknowledgement has been received from the database that the insert is successful right?
  2. The main thread(created when http request was received) could delegate and get a Future back, but is it still blocked until the onSuccess is trigged which gets triggered when the value is available, which means the db call was successful.

I am failing to understand how efficiently a HTTP server could be designed to maximize efficiency, what happens when few hundred requests hit a specific endpoint and how it is dealt with. I've been told that slick takes the best approach.

If someone could explain a successful http request lifecycle with future and without future, assuming there are 100 db connection threads.

druuu
  • 1,676
  • 6
  • 19
  • 36

3 Answers3

3

When a GET request is made, a success 200 means the data is written to the db successfully and not lost. Until the data is written to the server, the thread that was created is still blocking until the final acknowledgement has been received from the database that the insert is successful right?

The thread that was created for the specific request need not be blocked at all. When you start an HTTP server, you always have the "main" thread ongoing and waiting for requests to come in. Once a request starts, it is usually offloaded to a thread which is taken from the thread pool (or ExecutionContext). The thread serving the request doesn't need to block anything, it only needs to register a callback which says "once this future completes, please complete this request with a success or failure indication". In the meanwhile, the client socket is still pending a response from your server, nothing returns. If, for example, we're on Linux and using epoll, then we pass the kernel a list of file descriptors to monitor for incoming data and wait for that data to become available, in which we will get back a notification for.

We get this for free when running on top of the JVM due to how java.NIO is implemented for Linux.

The main thread (created when http request was received) could delegate and get a Future back, but is it still blocked until the onSuccess is trigged which gets triggered when the value is available, which means the db call was successful.

The main thread usually won't be blocked, as it is whats in charge of accepting new incoming connections. If you think about it logically, if the main thread blocked until your request completed, that means that we could only serve one concurrent request, and who wants a server which can only handle a single request at a time?

In order for it to be able to accept multiple request, it will never handle the processing of the route on the thread in which it accepts the connection, it will always delegate it to a background thread to do that work.

In general, there are many ways of doing efficient IO in both Linux and Windows. The former has epoll while the latter has IO completion ports. For more on how epoll works internally, see https://eklitzke.org/blocking-io-nonblocking-io-and-epoll

Yuval Itzchakov
  • 146,575
  • 32
  • 257
  • 321
1

First off, there has to be something blocking the final main thread for it to keep running. But it's no different than having a threadpool and joining to it. I'm not exactly sure what you're asking here, since I think we both agree that using threads/concurrency is better than a single threaded operation.

Future is easy and efficient because it abstracts all the thread handling from you. By default, all new futures run in the global implicit ExecutionContext, which is just a default threadpool. Once you kick of a Future request, that thread will spawn and run, and your program execution will continue. There are also convenient constructs to directly manipulate the results of a future. For example, you can map, and flatMap on futures, and once that future(thread) returns, it will run your transformation.

It's not like single threaded languages where a single future will actually block the entire execution if you have a blocking call.

When you're comparing efficiency, what are you comparing it to?

stan
  • 4,885
  • 5
  • 49
  • 72
  • When do you say a HTTP request is non-blocking and asynchronous? When a request is made, and lets assume the handler calls a function that returns a future, then we flatmap the future and if success, return 200. Until the future created in the handler is completed, the flatmap wont get called, hence the main thread that was created during request is still blocked. I do agree its got something to do with delegating to a different thread pool to release the http server's pool for taking more requests. I am not able to understand it. – druuu Nov 18 '17 at 19:46
  • I am also assuming or unaware of how a http request is handled by a server. I am assuming the request is delegated a specific thread each time for the full request-response cycle hence blocking it. – druuu Nov 18 '17 at 19:50
  • @jarvis11: " hence the main thread that was created during request is still blocked" - No, that's completely not accurate. It's not blocked because the future is run in another *thread*, which has no blocking impact on the main thread. There is nothing in a future that blocks the main thread other than the main thread blocking itself so it can keep running(and accepting more connections, hypothetically). – stan Nov 18 '17 at 19:58
  • @jarvis11 to give an example, the server can have a while loop that accepts new connections and starts a new future that handles the response. the while loop will continue as soon as the code to start the new future returns a promise. the main thread is not "blocked", it's just sleeping until it has something else to delegate – stan Nov 18 '17 at 20:03
  • Yes, I think I totally misunderstood how a request is handled by a http server. Once the promise is acquired, the thread(s) is(are) freed to take other requests. How many request can a single application instance(on a 512MB JVM) handle with non-blocking way of implementation(using futures)? – druuu Nov 18 '17 at 21:38
1

In general "non-blocking" may mean different things in different contexts: non-blocking = asynchronous (your second question) and non-blocking = non-blocking IO (your first question). The second question is a bit simpler (addresses more traditional or well-known aspect let's say), so let's start from it.

The main thread(created when http request was received) could delegate and get a Future back, but is it still blocked until the onSuccess is trigged which gets triggered when the value is available, which means the db call was successful.

It is not blocked, because Future runs on different thread, so your main thread and thread where you execute your db call logic run concurrently (main thread still able to handle other requests while db call code of previous request is executing).

When a GET request is made, a success 200 means the data is written to the db successfully and not lost. Until the data is written to the server, the thread that was created is still blocking until the final acknowledgement has been received from the database that the insert is successful right?

This aspect is about IO. Thread making DB call (Network IO) is not necessary blocked. It is the case for old "thread per request" model, when thread is really blocked and you need create another thread for another DB request. However, nowadays non-blocking IO became popular. You can google for more details about it, but in general it allows you to use one thread for several IO operations.

Aleksandr Ivannikov
  • 81
  • 3