My secondary nodes on my replica set close after I initiate the replica set

Question

My project is an app for a service-based company, as well as a general-purpose full stack boilerplate/template. The app has been up and running for months in development. This issue occurred initially 2 days ago.

In order to start my replica set, I first open 6 separate command prompt windows as admin. I then set the dbpath etc. for the primary:

 mongod --dbpath "C:\Program Files\MongoDB\Server\5.0\replicaComplex2\rep01\data" --logpath "C:\Program Files\MongoDB\Server\5.0\replicaComplex1\rep01\log\mongod.log" --port 30000 --storageEngine=wiredTiger --journal --replSet jupiter_rep1

Then in a new terminal I set the config:

mongo --port 30000

rsconfig={_id:"jupiter_rep1",members:[{_id:0,host:"localhost:30000"}]}

I usually have to reconf in order to set the primary:

----reconf          
rsconf = rs.conf()          
rsconf.members = [{_id: 0, host: "localhost:30000"}]            
rs.reconfig(rsconf, {force: true})

I then initialize the replica set:

rs.initiate(rsconfig)

Then I go to a new terminal/prompt and set the dbpath for the other two nodes:

mongod --dbpath "C:\Program Files\MongoDB\Server\5.0\replicaComplex2\rep02\data" --logpath "C:\Program Files\MongoDB\Server\5.0\replicaComplex1\rep02\log\mongod.log" --port 30001 --storageEngine=wiredTiger --journal --replSet jupiter_rep1

And the same for the third node.

However, this is where I am running into the problem. At this point the secondary nodes close out before I can actually set them as secondary.

I first tried to close all the windows and restart the actual machine. Nope.

Second, I uninstalled mongo dB and reinstalled it. Nope.

Third, I started the two secondary nodes prior to the primary or initialization. When i start the primary and initialize the replica set the secondary nodes shut down.

I'm on windows... I also have the MongoDB Server stopped.

Input is appreciated!

Update* I didn't include the log file error in my original question. They are separated for easier reading.

{"t":{"$date":"2022-04-15T16:05:44.353-05:00"},"s":"I", "c":"ROLLBACK", "id":21606, "ctx":"BackgroundSync","msg":"Finding common point"}

{"t":{"$date":"2022-04-15T16:05:44.353-05:00"},"s":"I", "c":"-",
"id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"ReadConcernMajorityNotAvailableYet: Read concern majority reads are currently not possible.","nextWakeupMillis":800}}

{"t":{"$date":"2022-04-15T16:05:44.377-05:00"},"s":"I", "c":"ROLLBACK", "id":21607, "ctx":"BackgroundSync","msg":"Rollback common point","attr":{"commonPointOpTime":{"ts":{"$timestamp": {"t":1649857370,"i":1}},"t":149}}}

{"t":{"$date":"2022-04-15T16:05:44.378-05:00"},"s":"F", "c":"ROLLBACK", "id":51121, "ctx":"BackgroundSync","msg":"Common point must be at least stable timestamp","attr":{"commonPoint":{"$timestamp": {"t":1649857370,"i":1}},"stableTimestamp":{"$timestamp": {"t":1649857964,"i":1}}}}

{"t":{"$date":"2022-04-15T16:05:44.378-05:00"},"s":"F", "c":"-",
"id":23091, "ctx":"BackgroundSync","msg":"Fatal assertion","attr":{"msgid":51121,"file":"src\mongo\db\repl\rollback_impl.cpp","line":1146}}

{"t":{"$date":"2022-04-15T16:05:44.378-05:00"},"s":"F", "c":"-",
"id":23092, "ctx":"BackgroundSync","msg":"\n\n***aborting after fassert() failure\n\n"}

Thanks!

Do you run all mongod's on the same machine? – Wernfried Domscheit Apr 15 '22 at 22:09 — Wernfried Domscheit, Apr 15 '22 at 22:09
Yes, in separate terminal windows – jboyer777 Apr 15 '22 at 23:04 — jboyer777, Apr 15 '22 at 23:04
OK, why do you run replica set on a single machine? – Wernfried Domscheit Apr 16 '22 at 05:01 — Wernfried Domscheit, Apr 16 '22 at 05:01
Each node just needs its own port. – jboyer777 Apr 19 '22 at 13:17 — jboyer777, Apr 19 '22 at 13:17

score 1 · Answer 1 · answered Apr 16 '22 at 09:06

There are several issues in your setup, thus I put them all in an answer, although it would be rather a comment.

Have a look at the logfiles C:\Program Files\MongoDB\Server\5.0\replicaComplex1\rep01\log\mongod.log and C:\Program Files\MongoDB\Server\5.0\replicaComplex1\rep02\log\mongod.log

They should show you the error message, I don't see any reason why the processes should stop.
Why do you have to have to reconf in order to set the primary? Does not make any sense. You should follow the Deploy a Replica Set tutorial.
I suggest to use a configuration file instead of command line options. Pay attention to the Windows Service Options
You should install MongoDB as a service, see mongod.exe, for example mongod --install --config C:\ProgramData\MongoDB\rep01\mongod.conf. The the services will start automatically at boot time.
Storage engine wiredTiger and enabled journal are default, you can skip these options.
As the name implies C:\Program Files\... is typically not the place where you store application data and logfiles. Usually in C:\Program Files\... you find only the binaries. Consider a different location, e.g. C:\ProgramData\MongoDB\... of C:\MongoDB\...

Wow, nice. Ill fix it up and see you back here! – jboyer777 Apr 16 '22 at 13:12 — jboyer777, Apr 16 '22 at 13:12

score 0 · Answer 2 · answered Apr 19 '22 at 16:10

At the suggestion of @Wernfried Domscheit, I created a mongo replica set service and, in the process, I did find the issue.

To create the replica service, I used the following guide.

Apparently, the log and dB path in my three config files all had the same link to the first node of the set. This is how it was months ago when I first copied and pasted the first config file to create the other two. So, I am thinking my external hard drive disconnected and some memory issue occurred. So due to the log paths being the same I got the log path error.

Thanks.

*Will switch to a Bluetooth set-up.

My secondary nodes on my replica set close after I initiate the replica set

2 Answers2