2

In our application we need to make around 10k REST api calls to 10k different endpoints URLs per minute. Earlier I was using synchronous model, but quickly realized that I can not scale beyond ~2k+ limit, so I am working on switching to an async model. Using HttpCore-NIO lib I could scale upto 5k or so, but beyond that I randomly get an error 'I/O reactor has been shut down' and the entire app basically stops processing requests. Don't see any stack trace either, which makes it extremely hard to debug.

So I am trying to evaluate what could be the best strategy/library to achieve this scale with Java as the programming language. Any suggestions on which libraries out there should I look into ?

mandar bapaye
  • 51
  • 1
  • 2

1 Answers1

0

If your machine itself is not stressed out on CPU, Network etc, then perhaps one thing that can help is horizontal scaling - if you can achieve 5K with 1 process, try firing requests from two processes; see what you get; how far you can horizontally scale. The extra coding to achieve horizontal scale should not be that much.

OR try trapping exceptions and ignoring them, to prevent I/O reactor shutdown. Experiment with what can be ignored. See example here https://hc.apache.org/httpcomponents-core-4.4.x/tutorial/html/nio.html#d5e477, search for "I/O reactor exception handling"

Experiment with thread count - setIoThreadCount

Make sure you are not doing too much processing with the HTTP responses; that would hold up the reactor.

Amit
  • 1,836
  • 15
  • 24
  • Thanks for the reply Amit. I was able to debug the Apache code and figured that Reactor shutdown was caused due to socket exhaustion. Was able to get around that. But facing a different issue now, where some of the connections get closed automatically sometimes. So looking into it now. thanks Once again. – mandar bapaye Sep 25 '15 at 22:50