0

i try to run a set of parallel thread in java. im creating these through a high order function as follows:

public static void parallelizedMap(Consumer<String> f, List<String> list, int count) {
    List<List<String>> parts = new ArrayList<List<String>>();
    final int N = list.size();
    int L = N / (count - 1);
    for (int i = 0; i < N; i += L) {
        parts.add(new ArrayList<String>(list.subList(i, Math.min(N, i + L))));
    }
    for (List<String> e : parts) {
        Runnable r = new Runnable() {
            public void run() {
                e.forEach(f);
            }
        };
        new Thread(r).start();
    }
}

this method is called every few minutes. it creates hundreds of thread after several minutes. every thread only runs for 20 seconds. but my debugging showed that they never terminate and therefor i get this Exception:

java.lang.OutOfMemoryError: GC overhead limit exceeded

thanks in advance

caesar
  • 1
  • 1
  • 3
    *every thread only runs for 20 seconds. but my debugging showed that they never terminate.* Those statements contradict each other. And we can't see what `f` is doing, so it's hard to offer much help. – shmosel Mar 20 '17 at 23:04
  • 2
    Some advice: Use meaningful variable names! Also, provide a complete example. – Christopher Schneider Mar 20 '17 at 23:27
  • if you use debugger, suspend any thread that must finish but didn't, and look at what statement it hangs. – Alexei Kaigorodov Mar 20 '17 at 23:57
  • hey thanks, f is checking a value in a database and the threads are waiting for some other thread to finish the database connection, i marked these methods synchronized. maybe this is leading to a deadlock situation, because there are threads that are created faster than they can connect to the database... is there a proper way to connect to mysql database without using synchronized? – caesar Mar 21 '17 at 09:12

2 Answers2

1

I do not think the problem are the threads. But if count is double or more than size of list:

final int N = list.size(); // N = 10, count = 22
int L = N / (count - 1);  // L = 10 / (22-1) = 0.476 / L = 0.4d = (int) 0;
for (int i = 0; i < N; i += 0) {
     parts.add(new ArrayList<String>(list.subList(i, Math.min(N, i + 0))));
}

The iteration is an endless-loop but the memory-usage grows every iteration.

Naturally its a OutOfMemory.

Grim
  • 1,938
  • 10
  • 56
  • 123
0

for me the solution was to give every thread an extra jdbc connection and dont let them share one. this resolved the deadlock und they terminate as expected.

caesar
  • 1
  • 1