9

Assuming I have 2 threads, one is writing to a FileOutputStream and one is reading from a FileInputStream.

The first thread has written x bytes.

When are the bytes considered as ready for reads?

The flush() method has an empty implementation on FileOutputStream, so flushing does not do anything.

My assumption is that once I wrote to the FileOutputStream bytes are ready to read.

An example of real life FileOutputStream and FileInputStream that will prove that after performing out.write() the bytes written are not ready to be consumed by in.read() will be much appreciated. A proof of the opposite will be even more appreciated.

galusben
  • 5,948
  • 6
  • 33
  • 52
  • 4
    I think this depends on the filesystem. Try using a PipedStream or something like that if you need to send data from one thread to another. – devgianlu Feb 26 '19 at 15:51
  • My use case is a little different and PipedStream is not what I need. – galusben Feb 26 '19 at 15:53
  • Could you share some of the code? Writing to a file can't be the only solution. – devgianlu Feb 26 '19 at 15:54
  • 2
    "If the intended destination of this stream is an abstraction provided by the underlying operating system, for example a file, then flushing the stream guarantees only that bytes previously written to the stream are passed to the operating system for writing; *it does not guarantee that they are actually written to a physical device such as a disk drive.*" From [`OutputStream`'s documentation](https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/io/OutputStream.html#flush()). – Federico klez Culloca Feb 26 '19 at 15:56
  • At the end of the process I must have the file written to disk, but during the time I write it, I need to read it from multiple threads. Those threads can start at any point of time, so I can not buffer, or keep in memory the bytes written. – galusben Feb 26 '19 at 15:57
  • Supposing that you have `flush()`ed the output, the bytes written are available to be read when the underlying operating system says they are. That some particular `FileOutputStream` implementation you've examined has an empty implementation of `flush()` does not mean you can assume that the flush is unneeded in general, and you also have no specific promises about when the bytes will be readable. – John Bollinger Feb 26 '19 at 15:58
  • @JohnBollinger - so you are saying that the writing thread can never assume the bytes have been written? – galusben Feb 26 '19 at 16:01
  • Not exactly, @gba. I'm saying that the writing thread cannot assume anything about when the bytes written can be read back. I'm mincing words a bit here because there are multiple applicable senses of the word "written". Java hands off the bytes to native code, native code hands them off to the OS, the OS hands them off to the device, and, for a regular file, the device records them on persistent media. These are not necessarily all synchronous, and it is system dependent how far the bytes need to go through that chain before they can be read back. – John Bollinger Feb 26 '19 at 16:11
  • @JohnBollinger "That some particular FileOutputStream implementation you've examined has an empty implementation of flush()" - isn't it more like **all** vendor implementations of FileOutputStream **must** not do anything in `flush()` because the documentation says so? – DodgyCodeException Feb 26 '19 at 16:11
  • Yes, @DodgyCodeException, you are correct. That a particular implementation has an empty `flush()` is indeed not more broadly indicative, but that the docs say that `FileOutputSream` inherits `flush()` from `OutputStream`, whose `flush()` is documented to do nothing, is relevant. In fact, `FileOutputStream` should *not* have any `flush()` method of its own at all. I'm nevertheless inclined to `flush()` anyway, for documentary purposes if nothing else. – John Bollinger Feb 26 '19 at 16:17
  • I would consider the bytes ready to be read by relying on NIO2's `WatchService`. Rather than try to figure out when the writing's done, I'll just check when the file has been created in a directory via the `ENTRY_MODIFY` Watch Key so I *know* the file is ready to be read. – Carlos Feb 26 '19 at 16:30
  • 1
    Consider using a [RandomAccessFile](https://docs.oracle.com/javase/8/docs/api/java/io/RandomAccessFile.html) open for both input and output, instead of separate InputStream and OutputStream. – DodgyCodeException Feb 26 '19 at 16:44
  • Thanks @DodgyCodeException, if I will use RandomAccessFile, it will lead me to the same question, when will the new bytes be visible for other threads? – galusben Feb 26 '19 at 20:05
  • @gba use the **same** RandomAccessFile object in all your threads. With suitable synchronization, of course, such as ReentrantReadWriteLock. – DodgyCodeException Feb 27 '19 at 12:01
  • 1
    I got my solution here: https://stackoverflow.com/q/12741937/2165342 – galusben Feb 27 '19 at 19:03

0 Answers0