Running a compiler in parallel with java.utils.concurrent

Question

I am making a compiler for a language in Java and I want it to compile many files in parallel.

I have my class Compiler.java that has the constructor Compiler(String fileName) and the method compile() So to compile a single file in my main all I do is :

Compiler c1 = new Compiler("file1.c");
c1.compile();

What I want to do is for a list of files ( lets say ["file1.c", "file2.c", "file3.c"] ) is executing the c1.compile(), c2.compile(), c3.compile() in parallel (c'i' being the compiler for file'i' ).

I am still a beginner in Java.util.concurrent. In C I just fork or use the POSIX thread library with the join method. But in Java I see there is more to it with things called thread pools etc. Any help will be much appreciated.

Thread pools are not unique to Java. Although, in some languages, the _standard_ library has one or more thread pool types, whereas in C or C++, you still either have to use a 3rd-party library, or else roll your own. — Solomon Slow, Dec 28 '21 at 23:44
The idea of a thread pool is to avoid the overhead of creating a thread and tearing it down once per request in a server-ish environment by re-using them repeatedly. For a compiler, this probably doesn't apply. Just create and exit the threads as needed. — Gene, Dec 28 '21 at 23:53

Solomon Slow · Accepted Answer · 2021-12-29T00:13:16.260

You don't have to use a thread pool in Java if all you want is a thread.* (Assuming you want to use threads at all **) Pretty much the simplest way to create a thread looks like this:

    Thread t = new Thread(() -> {
        ...code to be executed in the new thread goes here...
    });
    t.start();
    ...do other stuff concurrently with the new thread...
    try {
        t.join();
    } catch (InterruptedException ex) {
        // If your program doesn't use interrupts then this should
        // never happen. If it happens unexpectedly then, Houston! We
        // have a problem...
        ex.printStackTrace();
    }

* You might want to use a thread pool if your program otherwise would create many short-lived threads. The purpose of a thread pool, just like any other kind of "xxxx pool," is to re-use threads instead of continually creating and destroying them. Creating and destroying threads is relatively expensive compared to the cost of the tasks that some programs want to run in those threads.

Pretty much the simplest way to use a thread pool looks like this:

import java.util.concurrent.Executors;
import java.util.concurrent.ExecutorService;

    final int N_THREADS = ...however many worker threads you think you need...;
    final int PRACTICALLY_FOREVER = 999;

    ExecutorService thread_pool = Executors.newFixedThreadPool(N_THREADS);
    while (...still have work to do...) {
        thread_pool.submit(() -> {
            ...task to be executed by a worker thread goes here...
        });
    }
    thread_pool.shutdown();
    try {
        thread_pool.awaitTermination(PRACTICALLY_FOREVER, TimeUnit.DAYS);
    } catch (InterruptedException ex) {
        // If your program doesn't use interrupts then this should
        // never happen...
        ex.printStackTrace();
    }

** Some people think that threads are old-fashioned and/or low-level. Java has a whole other model for concurrency. You might want to take some time to learn about parallel streams.

Running a compiler in parallel with java.utils.concurrent

1 Answers1