"Persistent streams" for Java FileWriter?

Question

When I want to write Java code for writing text to a file, it usually looks something like this:

File logFile = new File("/home/someUser/app.log");
FileWriter writer;

try {
    writer = new FileWriter(logFile, true);

    writer.write("Some text.");

    writer.close();
} catch (IOException e) {
    e.printStackTrace();
}

But what if I'm doing a ton of write operations back-to-back? What if I'm logging dozens, hundred, even thousands of write operations per second?

In the same way that databases (annd JDBC) allow "persistent connections" (connections that remain open across multiple calls), is there a way to have "persistent streams" that do not need to be opened/closed across multiple write(String) invocations? If so, how would this work? What pitfalls/caveats would I have to be aware of? Thanks in advance!

Answer 1

If you look at this implementation

class Logger {
    private final BufferedWriter w;

    public Logger(final File file) throws IOException {
        this.w = new BufferedWriter(new FileWriter(file));
        LoggerRegistry.register(this);
    }

    public void log(String s) throws IOException {
        synchronized (this.w) {
            this.w.write(s);
            this.w.write("\n");
        }
    }

    public void close() throws IOException {
        this.w.close();
    }
}

the file is kept open.

If you have multiple threads, you have to synchronize in the write method (but this must be considered in any case).

There are possibly these problems, if the file remains open:

• In theory, you could run out of file handles. These might be limited (see ulimit -a on Linux systems for example): Each logger consumes one handle.

• If you use a FileWriter without buffering, you have I/O calls for each invocation of write. This could be quite slow.

• If you use a BufferedWriter on top of a FileWriter, you have to make sure that it gets closed properly at the end of your program, otherwise the remaining content in the buffer might not be written to the disk. So you need a try/finally block around your program which must close all loggers correctly.

Therefore you need to register all loggers. This is a simplified version (it is not thread-safe):

class LoggerRegistry {
    private final static List<Logger> loggers = new ArrayList<Logger>();

    public static void register(Logger l) {
        loggers.add(l);
    }

    public static void close() {
        for (Logger l : loggers) {
            try {
                l.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    }
}

and use this in your main program like this:

public static void main(String[] args) throws IOException {
    try {
        final Logger l = new Logger(new File("/tmp/1"));
        l.log("Hello");

        // ... 

    } finally {
        LoggerRegistry.close();
    }
}

If you have a web application, you could call the close method in a ServletContextListener (method contextDestroyed).

The biggest performance gain is probably the BufferedWriter. This advantage gets lost, if you open/close it for each write operation, since close has to call flush. So the combination of an open file together with buffering will be quite fast.

Answer 2

I am not sure how the logger frameworks manage the file connections, but one way I can suggest you is to cache the fileWriter in a static instance, without closing it. In this way, your program would hold live reference to the file and will not create new connection to write again.

One disadvantage of this approach, is that file will be held by FileWriter even if there are no write activities. This means that, the FileWriter will lock the file and while its locked, no other process can write on the file.