2

Due to cephs poor performance on NVME i'm experimenting with OCFS2 on DBRD again.

DBRD appears to initially have been built around the idea notifying an application of a hardware failure, and the application taking appropriate steps to move access to a new primary. This is not a useful pattern for our case.

DRBD 9 mentions a way based on quorums:

https://linbit.com/drbd-user-guide/drbd-guide-9_0-en/#s-configuring-quorum

if i understand correctly, loss of quorum (i.e. current node is in the minority partition) results in IO being frozen until quorum is reestablished. This is exciting news, as we have good experience with ceph recovering this way without intervention.

the DRBD manual sadly then mentions in section 12

https://linbit.com/drbd-user-guide/drbd-guide-9_0-en/#ch-ocfs2

All cluster file systems require fencing – not only through the DRBD resource, but STONITH! A faulty member must be killed.

which is the opposite of what it says earlier in quorum docs.

i'm not sure why it would require fencing anyway if quorum is supposed to already prevent the minority partition from executing write I/O. Is this because quorum does not actually work with dual-primary?

aep
  • 177
  • 2
  • 10

1 Answers1

3

Dual primary is limited to only two nodes. See this warning in the DRBD user's guide.

In DRBD 9.0.x Dual-Primary mode is limited to exactly 2 Primaries for the use in live migration.

Quorum requires 3 nodes or more. Thus you cannot use quorum in conjunction with dual-primary mode.

Dok
  • 1,158
  • 1
  • 7
  • 13
  • hmm. i was assuming a quorum member does not need to be primary. in fact it explicitly mentions using diskless members – aep Mar 14 '23 at 16:31
  • 1
    Perhaps the warning in the documentation could be worded a bit more clearly. The key word here is _exactly_ With dual-primary you a limited to _exactly_ two primaries. Not two primaries and a third diskless node. With a single Primary in an active-passive configuration, quorum is a perfectly valid solution to prevent data divergence and split-brains. With dual-primary and cluster aware file systems you do require STONITH. Do you need concurrent access to the DRBD devices on both nodes? – Dok Mar 14 '23 at 22:24
  • I was hoping to replace ceph, which is quorum based AND allows write from all nodes. If i understand correctly, if you only have one primary, you need another system on top to direct IO to that primary. i suppose that's what stonith partially solves. Anyway, i see that DRBD simply doesnt match our use case. thanks alot for the clear answer – aep Mar 15 '23 at 10:26