1

I wrote Thread pool class referring http://www.informit.com/articles/article.aspx?p=30483&seqNum=5

Environment: Windows7 4 cp

Executed my program with 70,000 Thread in Windows 7, under JDK 1.5 it went through successfully. Not used vm arguments.

The same Code i tried to execute with 5,000 Thread in Linux enterprise edition which is under Virtual Box with 4GB base memory. with vm arguments -xms512m -xmx1024m. It executes till 2156 threads and throws an exception

Exception in thread "main" java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:597) at testthreadpool.ThreadPool.(ThreadPool.java:38) at testthreadpool.TestThreadPool.main(TestThreadPool.java:16)

But the same code run perfectly in windows7.

May i know why this error occurs. Does this java code need 1GB memory to run Just 5,000 Threads?...

My actual requirement is to hold a ThreadPool with 10,000 Workthread.

user500796
  • 11
  • 1
  • 2

4 Answers4

4

My actual requirement is to hold a ThreadPool with 10,000 Workthread.

I think you need to revisit your requirement. That in no way is a good idea, and is catastrophic to performance.

Yann Ramin
  • 32,895
  • 3
  • 59
  • 82
  • The Constraint is not to use concurrency package. Need to use only palin Java Thread. – user500796 Nov 09 '10 at 03:46
  • May be bad idea. But there is no restriction on creating number of Threads and keep it in runnable mode when i give enough memory space. Its fails around 2156. – user500796 Nov 09 '10 at 03:46
  • Curious, why can't you use `java.util.concurrent`? – Jeremy Nov 09 '10 at 04:00
  • @user500796: The problem is not the use of plain Thread, but the fact that you are using 10,000 of them. Threads are not free, even if never used. There is no benefit to using more than a handful for performance reasons, and performance will be terrible if you are constantly context switching 10,000 threads. Consider using non-blocking IO, coroutines, or other mechanisms if you are trying to scale. – Yann Ramin Nov 09 '10 at 04:14
  • As we are dealing with low-latency app, we are expecting around max 7020 request per second to the worst case.All that request need to be processed and dispatched soon as possible. No delay accepted. – user500796 Nov 09 '10 at 08:43
0

As @Yann points out, using 10,000 threads is a really bad idea ... unless you have a machine with thousands of cores. You should take a serious look at your application design.

In the short term, try tuning the default thread stack size with the -Xss... JVM parameter. Also note that stacks are not allocated in heap memory, so your -Xms512m -Xmx1024m option is not reserving space for stacks. On the contrary, it is reserving space that cannot then be used for stacks.

Finally, there may be other things (other than memory for thread stacks) that will limit the number of threads that your application can create.

Stephen C
  • 698,415
  • 94
  • 811
  • 1,216
  • Guys that was really helpful. I did tuning and able to create 6000 for testing purpose. As advised we thought of reducing Thread count. – user500796 Nov 23 '10 at 02:07
0

Threads require a stack, that has to have an initial size. For threads, the initial stack size is by default the stack size reource limit, as shown by ulimit -s, but can be changed by a call to pthread_attr_setstacksize(). (See this other SO question).

Community
  • 1
  • 1
ninjalj
  • 42,493
  • 9
  • 106
  • 148
0

Are you on 64-bit?

Don't expect a 32-bit machine to be able to run lots of threads. You may also wish to tweak the stack size. Starting lots of threads uses lots of memory for stacks, and you can't get around that unless you can tolerate smaller stacks.

Checking on x86_64, Linux seems to default to 8M stacks, which means that 1k threads takes 8G stack, so you really want to be careful with that.

MarkR
  • 62,604
  • 14
  • 116
  • 151