Why does PW=all and PR=all not give strong consistency in Riak?

Question

Suppose we have a 5 machine cluster with n_val = 3. Why does setting PW=3 and PR=3 for writes and reads not guarantee strong consistency?

Answer 1

Writing a key using PW=3 writes the value to 3 primary vnodes. But the success is returned once the vnode receives the value to be written, i.e. before it is persisted.

After acknowledging the value, the vnode must still commit it to the appropriate backend store, any occurrence that interrupts this would result in a this vnode retrieving the previous value (or notfound) on the next read.

Using the durable write option (DW=3) would cause the vnode to delay acknowledgment of the value until after it has been committed to the backend store. However this is still not quite a guarantee, because the backend does not ensure cache is flushed after each value is written. A failure could still interrupt the value being written to disk. This is a decidedly smaller window than with PW=3, but still not a guarantee.

Granted the likelihood of this occurring on all 3 primary nodes such that the next PR=3 ready succeeds but returns the the wrong value is quite remote, but not nonexistant.

There is also the possibility of sibling values from simultaneous writes. If the first successful read after a successful write returns a set of sibling values instead of the last written value, is that strong consistency?