How mongo db replicaset works in Azure

Question

I have deployed the Mongo DB replica set using this template mongodb-replica-set-centos.

Mongo DB VM 1 (primary):

ps aux | grep mongo

root      10161  0.7  0.5 797140 40900 ?        SLl  05:18   0:05 mongod --dbpath /var/lib/mongo/ --replSet repset --logpath /var/log/mongodb/mongod.log --fork --config /etc/mongod.conf
sshuser   10347  0.0  0.0 112640   960 pts/0    S+   05:29   0:00 grep --color=auto mongo

Mongo DB database:-

 mongo -u mongoadmin -p mongoadmin admin
MongoDB shell version: 3.2.19
connecting to: admin
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
    http://docs.mongodb.org/
Questions? Try the support group
    http://groups.google.com/group/mongodb-user
Server has startup warnings: 
2018-03-23T05:18:21.137+0000 I CONTROL  [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2018-03-23T05:18:21.137+0000 I CONTROL  [initandlisten] 
repset:PRIMARY> rs.status()
{
    "set" : "repset",
    "date" : ISODate("2018-03-23T07:38:45.694Z"),
    "myState" : 1,
    "term" : NumberLong(1),
    "heartbeatIntervalMillis" : NumberLong(2000),
    "members" : [
        {
            "_id" : 0,
            "name" : "52.170.83.3:27017",
            "health" : 1,
            "state" : 1,
            "stateStr" : "PRIMARY",
            "uptime" : 8426,
            "optime" : {
                "ts" : Timestamp(1521782318, 3),
                "t" : NumberLong(1)
            },
            "optimeDate" : ISODate("2018-03-23T05:18:38Z"),
            "electionTime" : Timestamp(1521782318, 1),
            "electionDate" : ISODate("2018-03-23T05:18:38Z"),
            "configVersion" : 2,
            "self" : true
        },
        {
            "_id" : 1,
            "name" : "10.0.1.5:27017",
            "health" : 1,
            "state" : 2,
            "stateStr" : "SECONDARY",
            "uptime" : 8407,
            "optime" : {
                "ts" : Timestamp(1521782318, 3),
                "t" : NumberLong(1)
            },
            "optimeDate" : ISODate("2018-03-23T05:18:38Z"),
            "lastHeartbeat" : ISODate("2018-03-23T07:38:45.538Z"),
            "lastHeartbeatRecv" : ISODate("2018-03-23T07:38:42.546Z"),
            "pingMs" : NumberLong(1),
            "configVersion" : 2
        }
    ],
    "ok" : 1
}

Mongo DB VM 2 (secondary):

ps aux | grep mongo

root      10115  0.4  0.5 447908 37892 ?        SLl  05:11   0:17 mongod --dbpath /var/lib/mongo/ --config /etc/mongod.conf --replSet repset --logpath /var/log/mongodb/mongod.log --fork
sshuser   10269  0.0  0.0 112640   960 pts/0    S+   06:21   0:00 grep --color=auto mongo

Mongo DB database:-

mongo -u mongoadmin -p mongoadmin admin
MongoDB shell version: 3.2.19
connecting to: admin
2018-03-23T07:38:54.311+0000 E QUERY    [thread1] Error: Authentication failed. :
DB.prototype._authOrThrow@src/mongo/shell/db.js:1441:20
@(auth):6:1
@(auth):1:2

exception: login failed

Mongo DB VM 3(secondary):-

ps aux | grep mongo

root      10122  0.6  0.5 795472 40420 ?        SLl  05:12   0:26 mongod --dbpath /var/lib/mongo/ --config /etc/mongod.conf --replSet repset --logpath /var/log/mongodb/mongod.log --fork
sshuser   10381  0.0  0.0 112640   960 pts/0    S+   06:21   0:00 grep --color=auto mongo

Mongo DB database:-

 mongo -u mongoadmin -p mongoadmin admin
MongoDB shell version: 3.2.19
connecting to: admin
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
    http://docs.mongodb.org/
Questions? Try the support group
    http://groups.google.com/group/mongodb-user
Server has startup warnings: 
2018-03-23T05:12:19.613+0000 I CONTROL  [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2018-03-23T05:12:19.613+0000 I CONTROL  [initandlisten] 
repset:SECONDARY> rs.status()
{
    "set" : "repset",
    "date" : ISODate("2018-03-23T07:39:04.009Z"),
    "myState" : 2,
    "term" : NumberLong(1),
    "heartbeatIntervalMillis" : NumberLong(2000),
    "members" : [
        {
            "_id" : 0,
            "name" : "52.170.83.3:27017",
            "health" : 1,
            "state" : 1,
            "stateStr" : "PRIMARY",
            "uptime" : 8425,
            "optime" : {
                "ts" : Timestamp(1521782318, 3),
                "t" : NumberLong(1)
            },
            "optimeDate" : ISODate("2018-03-23T05:18:38Z"),
            "lastHeartbeat" : ISODate("2018-03-23T07:39:02.571Z"),
            "lastHeartbeatRecv" : ISODate("2018-03-23T07:39:03.573Z"),
            "pingMs" : NumberLong(1),
            "electionTime" : Timestamp(1521782318, 1),
            "electionDate" : ISODate("2018-03-23T05:18:38Z"),
            "configVersion" : 2
        },
        {
            "_id" : 1,
            "name" : "10.0.1.5:27017",
            "health" : 1,
            "state" : 2,
            "stateStr" : "SECONDARY",
            "uptime" : 8806,
            "optime" : {
                "ts" : Timestamp(1521782318, 3),
                "t" : NumberLong(1)
            },
            "optimeDate" : ISODate("2018-03-23T05:18:38Z"),
            "infoMessage" : "could not find member to sync from",
            "configVersion" : 2,
            "self" : true
        }
    ],
    "ok" : 1
}

Questions:-

Why I am unable to login into Mongo DB VM 2?
After I shutdown Mongo DB VM 3, will Mongo DB VM 2 acts as a secondary node?
If I shutdown Mongo DB VM 1, Will any one of secondary node act as a primary node?

Did you tried anything by yourself to get answers to questions? — Saleem, Mar 23 '18 at 09:34
I did shutdown Mongo DB VM 3, I don't see Mongo DB VM 2 acts as a secondary node — Galet, Mar 23 '18 at 09:47
I don’t see third node in replica set. There are only two modes so not enough members for election. — Saleem, Mar 23 '18 at 09:54
There are 3 Mongo VM. one is primary and other 2 nodes are secondary. — Galet, Mar 23 '18 at 10:33
I see this error "Not starting an election, since we are not electable due to: Not standing for election because I cannot see a majority (mask 0x1)" in /var/log/mongodb/mongod.log after I shutdown one secondary node — Galet, Mar 23 '18 at 11:39
@Saleem Did you know how to solve this error "Not starting an election, since we are not electable due to: Not standing for election because I cannot see a majority (mask 0x1" — Galet, Mar 26 '18 at 05:37

score 0 · Answer 1 · answered Apr 06 '18 at 09:01

All three questions are answered by the same fact: DB VM2 is not part of the replica set. It's clear from the rs.status() information that only two nodes are registered as part of the replica set, VM1 and VM3.

The implications are that:

On DB VM2, it is not part of the replica set so it does not have the authentication credentials you are trying to log in with
No, DB VM2 will not act as a secondary node; because it is not part of the replica set
In the current setup, with only 2 nodes in the replica set, if you shut down either node (VM1 or VM3) then the other node will not elect itself primary, because it cannot command a majority in an election.

Take a look at the docs on Replica Set Elections to understand what the majority is and why it matters; and take a look at DB VM2 to understand why it is not part of your replica set. Did you ever actually add it?

How mongo db replicaset works in Azure

1 Answers1