6

I've been learning about the close-to-open policy of NFS, which causes each file to be flushed to the server as it's closed, to ensure consistency across clients. (See http://docstore.mik.ua/orelly/networking_2ndEd/nfs/ch07_04.htm.) This leads to a big performance hit when trying to write many small files.

I know about the "async" export option, obviously, but there is also a "nocto" client mount option, which is supposed to disable the close-to-open mechanism for that client. As far as I can tell, this is supposed to stop the client from flushing the file when it closes (as well as not bothering to check cache consistency when it opens). However, this seems to have no effect: the client is still flushing the file to the server on close, causing massive wait-io.

Does anyone have any idea why "nocto" doesn't have the effect I was hoping it would? The "async" option works as expected, but it's more important to me that the client cache is correct in this case, and it just bugs me.

Example: a set of diskless nodes share a remote root, which is occasionally updated from one of the nodes. It is not critical that each file is flushed immediately after it is closed, as no other node is trying to write to the same file. However, it is more critical that if the server crashes whilst updating a set of packages, the client is aware of what data has not yet been written to the server's disk, so that it can try again once the server is up again. Using the "async" option, this scenario might result in data loss (because the server lies to the client about data being flushed to disk), whereas disabling close-to-open (and using "sync" instead of "async") should in theory provide the same performance benefit without the potential data loss (because several file writes would be buffered and flushed to the server together). The server and other clients would see a slightly outdated view of the filesystem (by a couple of seconds). This seems reasonable to me.

To put it really simply, "async" does server-side buffering, which speeds things up a lot. What I'm expecting is that "nocto" should do client-side buffering, with similar speed boosts, at the cost of some lag in data appearing on other clients.

Paul Gideon Dann
  • 181
  • 2
  • 6
  • 1
    "any idea why 'nocto' doesn't have the effect I was hoping it would?" Did you test it during the day? It only works at night. ;) – Brian Cain Aug 20 '13 at 15:27

1 Answers1

1

From the nfs(5) man page:

If the nocto option is specified, the client uses a non-standard heuristic to determine when files on the server have changed.

Using the nocto option may improve performance for read-only mounts, but should be used only if the data on the server changes only occasionally. The DATA AND METADATA COHERENCE section discusses the behavior of this option in more detail.

Cristian Ciupitu
  • 6,396
  • 2
  • 42
  • 56
kofemann
  • 4,626
  • 1
  • 25
  • 30
  • 1
    Thanks; I had already read the man page, including the BSD man page, which (if I remember correctly) describes behaviour more similar to what I expect. I've added an example to better illustrate my point. – Paul Gideon Dann May 29 '13 at 14:20