0

I have a PSA setup (mongod v5.0.3) running on EC2 r5a series instances.

My goal is to migrate these servers to Amazon Graviton instances (r6g) and Upgrade to mongod v6.0.3.

Current Setup:

P - r5a - mongod v5.0.3
S - r5a - mongod v5.0.3
A - t3a - mongod v5.0.3

Step1 - Adding a Secondary node to the replica set:

  • Installed mongod v6.0.3 in a new r6g instance and added it to the replica set.
  • Initial sync completed and state changed to SECONDARY.
  • In the logs of Primary, we can see
Beginning automatic reconfig to remove ‘newlyAdded’ config field

Removed ‘newlyAdded’ config field

A query on local.system.replset collection shows no newlyAdded field on this server. Everything is fine here.

Step2 - Adding another Secondary node to the replica set:

  • Installed mongod v6.0.3 in another new r6g instance and added it to the replica set.
  • Initial sync completed and state changed to SECONDARY.
  • In the logs of Primary, we can see
Beginning automatic reconfig to remove ‘newlyAdded’ config field
Beginning automatic reconfig to remove ‘newlyAdded’ config field
Beginning automatic reconfig to remove ‘newlyAdded’ config field
Beginning automatic reconfig to remove ‘newlyAdded’ config field
...
  • A query on local.system.replset collection shows newlyAdded field on this server. And it isn’t getting removed.
  • This server is forever stuck in “Beginning automatic reconfig to remove newlyAdded config field”.

Server is in SECONDARY state. But newlyAdded field isn’t getting removed.

Summary is

  • Newly added member 1 got automatically reconfigured and newlyAdded field got removed
  • Any subsequent newly added member isn't getting automatically reconfigured and newlyAdded field isn't getting removed.

If a member is marked newlyAdded, it can not participate in voting.

Because of this I’m unable to remove old servers running v5.0.3

Any leads and suggestions are much appreciated.

Searching on MongoDB documentation and google doesn't give much information on how to fix this problem.

Hence, we removed this member and re-added. Same issue!

If we add more members, same issue re-occurs for those nodes too.

Ultimately we had to remove all new members from the replica set and we are stuck.

We want to add new members since we want to migrate our current ec2 instance types of existing members.

Raveendra Bhat
  • 1
  • 1

0 Answers0