57

I would like to know the internals of the tomcat NIO connector. How exactly are threads used when we create a servlet that implements CometProcessor?Is it still one thread per connection?

From what I read, the conversation goes like this

  1. Client connects to a servlet

  2. Servlet hangs on to the connection till there is any data available to the connected client

  3. When data is ready , the server writes to the httpResponse and flushes it. This actually disconnects the connection?

  4. Client sends another request which the server again hangs onto..

How many thread are used when this keeps happening?

Christopher Schultz
  • 20,221
  • 9
  • 60
  • 77
user1456150
  • 579
  • 1
  • 5
  • 3
  • To which version of Tomcat do you refer? This kind of behavior is changing in later versions of Tomcat 7 and in Tomcat 8. – Basil Bourque Apr 19 '14 at 11:15

4 Answers4

60

NIO and Comet are completely unrelated: you can mix-and-match them.

Using the NIO (or APR for that matter) connector allows you to handle more requests with fewer threads due to the threading model. See http://tomcat.apache.org/tomcat-7.0-doc/config/http.html#Connector_Comparison for a comparison between the Connectors.

Comet (and Websocket) have a completely different dispatch model which requires a different application architecture and achieves higher throughput in a different way.

The scenario you pose in your question is the typical blocking one-thread-per-request model. In step 4, the Java BIO connector (which is the default up through Tomcat 7) will continue to wait for additional requests on the existing connector -- for keepalive HTTP requests. If the client does not set Connection:close on the previous request and does not close the connection, the thread will hang until the keepalive timeout is reached. If you use the NIO connector, the thread will go back into the thread pool immediately after the response is sent and you won't "waste" a thread on keepalive requests that may never arrive.

Comet/Websocket works entirely differently by delivering a message to a specially-written servlet (and optional filters) and the threads are only used when there are messages to send or data to be written.

UPDATE 2016-08-19

Tomcat 8.5 and 9.0 have completely dropped the BIO connector. This is because many of the new APIs and technologies (e.g. Websocket) require non-blocking semantics, and building a non-blocking service on top of a blocking API is very very difficult. The code required to get the job done was making the rest of the Tomcat code very ugly, etc. and so the decision was made to drop the BIO connector completely. So for Tomcat 8.5 and onward, only NIO, NIO2, and the APR-based connectors are available.

Note that, also with Tomcat 8.5 and 9.0, support for Comet has been dropped. Uses of Comet should all be replaced with Websocket which is a more standard protocol.

Christopher Schultz
  • 20,221
  • 9
  • 60
  • 77
  • 4
    It sounds like NIO is a win-win, so why is BIO the default when APR is not available? – Jayen Sep 18 '14 at 01:17
  • 4
    @Jayen BIO is the default connector for Tomcat up through 7.0.x. In Tomcat 8, NIO is the [default connector](http://tomcat.apache.org/migration-8.html#Default_connector_implementation). – Christopher Schultz Sep 18 '14 at 21:17
  • Any reason it wasn't the default on tomcat 6 or 7 (with java 7)? – Jayen Sep 19 '14 at 19:22
  • 3
    @Jayen Yes, it was still experimental in Tomcat 6 and the decision to make NIO the default was made after Tomcat 7 was released. So, rather than changing the default connector in a point release and potentially causing all kinds of problems, the switch to a NIO-default was made in Tomcat 8. Anyone can explicitly change their connector to NIO at will... it's just the default that has changed and the NIO connector is quite stable at this point. – Christopher Schultz Sep 22 '14 at 17:26
  • I have a question about 'If you use the NIO connector, the thread will go back into the thread pool immediately after the response is sent and you won't "waste" a thread on keepalive requests that may never arrive.', it sound that in NIO, the keepalive is meaningless? – liam xu Feb 28 '19 at 09:22
  • @liamxu For NIO, the `keepalive` setting is much less important, at least when it comes to being able to serve concurrent *requests*. But, remember, that the duration of the keepalive timeout will allow clients to remain *connected* even if they are not actively making requests. This may tie-up your TCP/IP resources, so it's important not to leave this timeout very hing in a high-availability environment. – Christopher Schultz Mar 01 '19 at 16:42
4

NIO use fewer thread, it means that the tcp/ip port use is fewer.

You know the port is 1 to 65534, so we can say that NIO can reach a higher TPS (Transactions Per Second) than BIO

I tested both protocol :HTTP/1.1 and org.apache.coyote.http11.Http11NioProtocol with same web-project, same host, and same server.xml but the protocol.

Use jmeter for test.

I set 1000 thread to run request, when HTTP/1.1 in a few minutes, the host use port is more than 30000 and the TPS is 300 only!

When org.apache.coyote.http11.Http11NioProtocol, the max count of use port is never overstep 3000 and the tps is more than 1200!

Eric Alberson
  • 1,116
  • 1
  • 11
  • 23
toby941
  • 378
  • 2
  • 4
  • 2
    Please reread your post, it is very hard to understand. I made a few edits to help clear it up a bit. If you want to start a new paragraph you need 2 carriage returns. – Joshua Wilson Dec 03 '13 at 07:08
  • If you update it, post another comment that you did and I will review it again and upvote it if you fix it. :) – Joshua Wilson Dec 04 '13 at 03:06
  • @toby941 what is `TPS`? Unless obvious, mention full term before using an acronym. – Basil Bourque Apr 19 '14 at 11:13
  • 1
    Transactions Per Second –  May 09 '14 at 10:59
  • 3
    TPS measurements are highly usage-dependent. NIO also requires more resources (CPU, specifically) than BIO. The best way to decide which connector to use is to benchmark your own application and see how it performs under different conditions and configurations. – Christopher Schultz Sep 18 '14 at 21:20
  • 5
    Also note that port use is irrelevant, here: each connector, regardless of the type (BIO, NIO, NIO2, APR) uses a single TCP/IP port. – Christopher Schultz Feb 03 '15 at 20:51
  • 1
    Are your benchmarks reproducible, do you mind sharing? I am assuming you enabled keep-alive on Jmeter client and not doing much CPU work in the servlet that gives NIO a better TPS. – kisna Dec 14 '16 at 08:08
  • @kisna Benchmarks have been published and presented in a few of the "connector"-related [presentations listed](http://tomcat.apache.org/presentations.html) on the Tomcat web site. – Christopher Schultz Oct 05 '18 at 15:21
4

Adding late to this discussion - In this context of performance comparisons between blocking IO and Asynchronous NIO - an excellent read is "Old way of writing servers is new". In summary below thread per connection model was found to be better performing and easy to write as compared to the NIO version - contrary to the popular belief.

1

Here are two good articles on the NIO connector in case this helps someone considering the differences between BIO (request processing is bound to accept thread) & NIO (request processing is passed onto another worker thread) in Tomcat.

Mike Barlotta
  • 715
  • 4
  • 16