Referential transparency and mmap in Haskell

Question

I was hoping to use System.INotify and System.IO.MMap together in order to watch for file modifications and then quickly perform diffs for sending patches over a network. However, in the documentation for System.IO.MMap there's a couple of warnings about referential transparency:

The documentation states

It is only safe to mmap a file if you know you are the sole user. Otherwise referential transparency may be or may be not compromised. Sadly semantics differ much between operating systems.

The values that MMap returns are IO ByteString, surely when I use this value with putStr I'm expecting a different result each time? I assume that the author means that the value could change during an IO operation such as putStr and crash?

START-OF-EDIT: Come to think of it, I guess answer to this part of the question is somewhat obvious... If the value changes any time after it is unboxed it would be problematic.

do 
  v <- mappedValue :: IO ByteString
  putStr v
  putStr v  -- Expects the same value of v everywhere

END-OF-EDIT

Shouldn't it be possible to acquire some kind of lock on the mapped region or on the file?

Alternatively, would it be possible to write a function copy :: IO ByteString -> IO ByteString that takes a snapshot of the file in its current state in a safe way?

Sounds to me like what you want could be done as well with a simple pair of pipes. — leftaroundabout, Jul 01 '12 at 19:54
I'm "watching" the files in a directory in a somewhat generic fashion. If any tool touches them, the clients automatically get updated. — Rehno Lindeque, Jul 01 '12 at 21:34
Is there really no mmap library that offers something of type `IO (Ptr Word8)` or similar? — Daniel Wagner, Jul 01 '12 at 21:59
I never even thought to check. There's http://hackage.haskell.org/packages/archive/bindings-mmap/0.1/doc/html/Bindings-MMap.html and then there's the internal part of http://hackage.haskell.org/packages/archive/bytestring-mmap/0.2.2/doc/html/System-IO-Posix-MMap-Internal.html. Thanks so much. I'm still surprised at how large the Haskell ecosystem is sometimes. — Rehno Lindeque, Jul 01 '12 at 22:18
Might also be the same as using the ForeignPtr stuff in http://hackage.haskell.org/packages/archive/mmap/0.5.8/doc/html/System-IO-MMap.html — Rehno Lindeque, Jul 01 '12 at 22:25

score 9 · Accepted Answer · answered Jul 01 '12 at 18:52

I think the author means that the value can change even inside a lifted function that can view it as a plain ByteString (no IO).

The meory mapped file is a region of memory. It doesn't make much sense to copy its content back and forth, for performance reasons (otherwise one could just do plain old stream-based I/O). So the ByteString you are getting is live.

If you want to have a snapshot, just use a stream-based I/O. That's what reading a file does: creates a file snapshot in the memory! I guess an alternative would be using the ForeignPtr interface which does not carry the referential transparency warning. I'm not familiar with ForeignPtrs so I cannot guarantee it will work, but it looks promising and I would investigate it.

You can also try calling map id on your ByteString but it is not guaranteed you will get a copy distinct from the original.

Mandatory file locking, especially on Linux, is a mess that is better avoided. Advisory file locking is OK, except nobody is using it, so it effectively does not exist.

I guess that subconsciously I'm expecting a bit too much from my operating system. I wanted to treat the file, basically, like a really fast shared-memory cache between multiple processes, letting the OS take care of flushing changes to disk at will. Thinking about it more carefully I guess that seems fairly unlikely to work unless all the processes explicitly use a shared memory map. — Rehno Lindeque, Jul 01 '12 at 19:10
(See, I hoped to avoid incurring the latency involved in touching the physical disk...) — Rehno Lindeque, Jul 01 '12 at 19:22

Referential transparency and mmap in Haskell

1 Answers1