Buffer chaining and decorative principle Java I/O

Question

Yes I know what buffer is. But watch this:

BufferedWriter bufferedWriter = new BufferedWriter(new FileWriter("file.txt"));

How does buffering actually work here? The way I see it, we are buffering data in FileWriter buffer and not in BufferedWriter buffer. Because when buffer of BufferedWriter gets full it will send it to FileWriter buffer and it will be responsible for writing data?

Am I missing something? The way I see it: It looks as we are sipping water from a bigger container to a smaller one. So we end up pouring water from smaller one.

Simmilar example here:

Scanner scanner = new BufferedReader(new FileReader("file.txt"));
scanner.nextLine();

I have seen this everywhere. We actually end up reading from Scanner line by line and not from buffer and its 8k capacity. So what is the point of buffer here? We read line by line from file and not entire buffer at once. Is bufferedReader redundand here?

Please if someone can nicely explain this, I have been struggling for a long time.

Answer 1

Low level system calls to read and write data are optimized to transfer larger blocks at once. Buffering lets you take advantage of this. When you write single characters or short strings, they are all accumulated in a buffer, and written out as one large block when the buffer is full. When you read data, the read functions request to fill a large buffer, and then it returns data from that buffer.

You're right that wrapping buffered streams within other buffered streams is pointless: at best it achieves nothing, at worst it adds overhead as the data is needlessly copied from one buffer to the next. The buffer closest to the data source matters most.

On the other hand, nothing in the API specification says FileWriter and FileReader have buffers. In fact, it recommends you wrap FileWriter within a BufferedWriter and FileReader within a BufferedReader:

For top efficiency, consider wrapping an OutputStreamWriter within a BufferedWriter so as to avoid frequent converter invocations. For example:

Writer out
  = new BufferedWriter(new OutputStreamWriter(System.out));

(FileWriter is a subclass of OutputStreamWriter)

How does this work internally?

If you look at how FileWriter is implemented though, the story gets complicated because FileWriter does involve a buffer. Some of the details may depend on which version of Java you're using. In OpenJDK, when you create a BufferedWriter that decorates a FileWriter like:

BufferedWriter bufferedWriter = new BufferedWriter(new FileWriter("file.txt"));

you are creating a stack of objects like the following, where one object wraps the next:

BufferedWriter -> FileWriter -> StreamEncoder -> FileOutputStream

where StreamEncoder is an internal class, part of how OutputStreamWriter is implemented.

Now, when you write characters to the BufferedWriter instance, it first accumulates them in the BufferedWriter's own buffer. The inner FileWriter does not see any of the data until you have write enough data to fill this buffer (or call flush()).

When the BufferedWriter buffer becomes full, it writes the contents of the buffer to the FileWriter with a single call to write(char[],int,int). This transfer of a large data block is where the efficiency comes from: now FileWriter has a large block of data it can write to the file, and not individual characters.

Then it gets a little complicated: the characters have to be converted to bytes so that they can be written into a file. This is where FileWriter passes these data on to StreamEncoder.

The StreamEncoder class uses a CharsetEncoder to convert the block of characters to bytes all at once, and accumulates the bytes in a buffer of its own. When it's done, it writes the bytes to the innermost FileOutputStream, as one block. FileOutputStream then invokes operating system functions to write to an actual file.

What if you didn't use BufferedWriter?

If you write characters to the FileWriter directly, they get passed on to the StreamEncoder object, which converts them into bytes and stores in its private buffer, and not written directly to the FileOutputStream. This way, the internal implementation of FileWriter gives you some of the benefits of buffering. But this is not a part of the API specification so you shouldn't depend on it.

Also, every call to FileWriter.write will result in an invocation to the CharsetEncoder to encode characters into bytes. It's more efficient to encode large blocks of characters at once, writing single characters or short strings has a higher overhead.