1

My Java web application use NFS file system, I use FileOutputStream to open, write multiple chunks and then close the file.

From the profiler stats I found that stream.write(byte[] payload,int begin, int length) and even stream.flush() takes zero milliseconds. Only the method call stream.close() takes non-zero milliseconds.

It seems that java FileOutputStream's write() or flush() doesn't really cause NFS client to send data to NFS server. Is there any other Java class will make NFS client flush data in real time? or there is some NFS client tuning need to be done?

user1532146
  • 184
  • 2
  • 14

2 Answers2

1

You are probably running into Unix client-side caching. There are lots of details here in the O'Reilly NFS book.

But in short:

Using the buffer cache and allowing async threads to cluster multiple buffers introduces some problems when several machines are reading from and writing to the same file. To prevent file inconsistency with multiple readers and writers of the same file, NFS institutes a flush-on-close policy: All partially filled NFS data buffers for a file are written to the NFS server when the file is closed.

For NFS Version 3 clients, any writes that were done with the stable flag set to off are forced onto the server's stable storage via the commit operation.

NFS cache consistency uses an approach called close-to-open cache consistency - that is, you have to close the file before your server (and other clients) get a consistent up-to-date view of the file. You are seeing the downsides of this approach, which aims to minimize server hits.

Avoiding the cache is hard from Java. You'd need to set the file open() O_DIRECT flag if you're using Linux; see this answer for more https://stackoverflow.com/a/16259319/5851520, but basically it disables the client's OS cache for that file, though not the server's.

Unfortunately, the standard JDK doesn't expose O_DIRECT. as discussed here: Force JVM to do all IO without page cache (e.g. O_DIRECT) - essentially, use JNI youself or use a nice 3rd party lib. I've heard good things about JNA: https://github.com/java-native-access/jna ...

Alternatively, if you have control over the client mount point, you can use the sync mount option, as per NFS manual. It says:

If the sync option is specified on a mount point, any system call that writes data to files on that mount point causes that data to be flushed to the server before the system call returns control to user space. This provides greater data cache coherence among clients, but at a significant performance cost.

This could be what you're looking for.

Community
  • 1
  • 1
SusanW
  • 1,550
  • 1
  • 12
  • 22
  • I'm using NFS version 3 client, is there a mount option could be set to force flush to the server? – user1532146 Aug 16 '16 at 18:58
  • @user1532146 try the `sync` mount option. Checking here: http://man7.org/linux/man-pages/man5/nfs.5.html, I'll update the answer. Slightly nervous because I haven't tried it ... (and I'm desperately trying to figure out if it's NFSv3 or v4!) – SusanW Aug 16 '16 at 19:36
  • @user1532146 ok, it's supported in v3. – SusanW Aug 16 '16 at 19:55
  • 1
    Great! After knowing the pros and cons we choose not to use this option because of its performance penalty and we always use the same NFS client instance for the same file's reading and writing. – user1532146 Aug 17 '16 at 14:10
  • @user1532146 fair enough - it was starting to sound exciting! I was going to propose you could work around by writing to a local (tmp) file and then using a separate mechanism to append the local file to the central one - sort of like log consolidation. But it depends on what you're actually trying to do :-) Good luck! – SusanW Aug 17 '16 at 14:33
-1

Generally, Java's streams make no guarantee about the effects of flush apart from maybe the flushing of the buffers in the Java classes involved.

To overcome that limitation, Java NIO's Channels can be used, see e.g https://docs.oracle.com/javase/7/docs/api/java/nio/channels/FileChannel.html#force(boolean). However, if "the file does not reside on a local device then no such guarantee is made." And Java cannot make such a guarantee, because the underlying remote file system or protocol may not be able to provide that function at all. However, you should be able to achieve (almost) the same level of synchronization with force() that you'd get from a native O_DIRECT access that @SusanW mentioned.

JimmyB
  • 12,101
  • 2
  • 28
  • 44
  • So there is still no guarantee. Hard to see the point of the answer. – user207421 Aug 17 '16 at 12:37
  • IMHO, it perfectly answers the question "Why java FileOutputStream's [...] flush() doesn't [...] really send data to NFS server?" Hard to see why anybody except @EJP would downvote. – JimmyB Sep 28 '16 at 12:12
  • It doesn't answer the question about `FileOutputStream`'s *`.write()` or* `flush()` in any way, and doesn't provide a solution to the problem, because the problem isn't with Java, it is with NFS. And the statement about `flush()` not providing any guarantees is simply false. `FileOutputStream.flush()` is specifically documented, via inheritance, to do nothing. – user207421 Jan 20 '17 at 03:10