1

I have a MongoDB replica set with 3 nodes and 1 arbiter.

The nodes have priority 0, 1, and 10. The arbiter has priority 1. They all have 1 vote.

The priority 10 was PRIMARY, while the priority 0 and 1 were both SECONDARY.

I set the default write concern to {w: 1}.

As a test I cleanly shut off one of the SECONDARY, namely the priority 0 node, with db.shutdownServer().

What I expected to happen is that everything continued to operate normally. Since the primary was still up, and the write concern was 1 (meaning only the primary needs to acknowledge), then everything should still be ok.

However, what I observed is that sometimes a write query would work, while other times it would block forever. My system as a whole stopped working. As soon as I brought the secondary back up, it started working again normally.

Why is this happening? One of my co-workers said it's because the queries try to write to any one of the nodes, and if they pick the secondary that is down then it will fail, but that isn't my understanding from what the documentation says.

Is the replica set really supposed to fail if a SECONDARY goes down? Isn't the point of the replica set to solve precisely this issue?

Additional info: when I set the priority 0 node to also have 0 votes, shutting it down didn't affect anything. So it seems it having a vote contributes to the issue... but not sure what it could be.

Claudiu
  • 224,032
  • 165
  • 485
  • 680
  • 1
    This *may* be something for http://dba.stackexchange.com… – deceze Dec 08 '22 at 11:05
  • An arbiter is recommended only for even number of members, for 3 nodes it does not make much sense. Anyway, in general it should work. Some commands (e.g. `db.collection.renameCollection()`) set implicitly `{w: 'majority'}`, you cannot change that. However, 2 out of 3 (i.e. 3 data bearing nodes) is still the majority, so it should work. – Wernfried Domscheit Dec 08 '22 at 13:01
  • Looks like you are not the only one: [Mongo cannot write to primary](https://stackoverflow.com/questions/74716744/mongo-cannot-write-to-primary/74722395) Same questions: Do you use TLS/SSL? Which version do you use? What is the output of `rs.status()`? Do you see any warnings in SECONDARY log files? – Wernfried Domscheit Dec 08 '22 at 13:02

0 Answers0