How do cluster file systems avoid the myriad of possible race conditions?
I'm trying to get a grip on using a cluster file system in a Master-Master architecture. I'm thinking specifically about GlusterFS, so implementation details for it are welcome, but I'm hoping for a general answer.
A GlusterFS process comprises of stacked modular functional units called "translators". A locking translator is loaded as part of the server stack and operations that could race are synchronized by this translator.
By default, the gluster CLI generates configuration that includes the "features/locks" translator as part of the GlusterFS server stack.
Different cluster filesystems handle this in different ways. Fundamentally the problem is like handling any other shared data structure in a concurrent-access scenario (thread safe structures, for example). Data on disk access is orders of magnitude more latent and slower (less bandwidth) to access than data stored in RAM. As such, the algorithms employed to handle concurrency in a cluster filesystem will be tuned appropriately.
A "traditional" cluster filesystem like, say, VERTIAS Cluster Filesystem, uses a lock manager and a master/slave process (with a fencing mechanism) to handle keeping the on-disk metadata consistent.
Personally I haven't used GlusterFS and can't say that I'm familiar enough with the architecture to make intelligent comments. From what I'm gleaning from reading about it, GlusterFS seems more to be a fancy mechanism to "glue together" underlying non-clustered filesystems on individual storage nodes and not a "traditional" cluster filesystem (where multiple clients are directly accessing the same block devices). You can get some good details here (though I don't know how current these details remain in the most recent versions): http://www.raidinc.com/pdf/whitepapers/wp-gfs-architecture.pdf
