0

I have a MongoDB ReplicaSet in three-member primary-secondary-arbiter (PSA) architecture, and I have a Play (v2.8) web app that uses ReactiveMongo ver 1.0.10. The connection string looks like the following:

mongodb://<user>:<pass>@mongo-pri:27017,mongo-sec:27017,mongo-arb:27017/<db>?authenticationMechanism=scram-sha1&connectTimeoutMS=20000&rm.nbChannelsPerNode=2000&replicaSet=<replica-name>&w=1&rm.keepAlive=true

I have two questions:

  1. ReactiveMongo gives a warning that complaint the arbiter node is not queryable. I don't have account and data on the arbiter node and it is not supposed to be "login-able".

    [WARN] from reactivemongo.core.actors.MongoDBSystem in reactivemongo-akka.actor.default-dispatcher-8 - [Supervisor-2/Connection-1] Node Node[mongo-arb:27017: NonQueryableUnknownStatus<14689291898971648ns> (200/200/200 available connections), latency=6ms, authenticated={}] has been not queryable for at least 300s; Removing it from the set! Please check configuration and connectivity

  2. The primary and secondary node are in different premises and are connected via OpenVPN. Recently, I setup a batch to process some image files in the Play web app and I have created a separate execution context for this. I observed the many exceptions with ReactiveMongo, which the Play web app looks halt and I have to restart the app. Meanwhile I can query the primary node normally with Studio3T or mongo shell. I turn out removed the secondary and arbiter node as well as the replica set name from the connection string and now the system seems to be stable again. The secondary node is just for backup and I don't need a failover indeed. So, removing the secondary and arbiter node from connection string is ok for me. However, is it appropriate to do so?

reactivemongo.core.actors.Exceptions$InternalState:
        at reactivemongo.ChannelConnected(4cd9fd04, {{NodeSet Some(sonobrain-rs) Node[mongo-pri:27017: Primary<17536106174783815ns> (2000/2000/2000 available connections), latency=3s, authenticated={scoliouser@scolionet}] | Node[mongo-sec:27017: Secondary<17536106177898008ns> (1396/1396/2000 available connections), latency=17536120s, authenticated={scoliouser@scolionet}] | Node[mongo-arb:27017: NonQueryableUnknownStatus<17536106209721144ns> (1213/1213/2000 available connections), latency=17536119s, authenticated={}] }})(<time:17536121496542193>)
Joseph Hui
  • 573
  • 6
  • 11
  • Mostly probable that the issue is due to the VPN, with node advertising themselve with host or IP only accessible from "their" network. First check with node on the same network. Then make sure the node visibility is align with the network resolution. – cchantep Mar 16 '23 at 16:02
  • I believe the VPN is working fine. When I run "rs.status()" in mongo shell, the replication looks healthy. Moreover, I can connect to the secondary node from the primary node with mongo shell. One strange thing is that ReactiveMongo reported warnings "The node set is authenticated, but the primary is not available: Some(sonobrain-rs) -> Set(mongo-sec:27017), Set(mongo-pri:27017)" and "reactivemongo.core.actors.Exceptions$InternalState: at reactivemongo.ScramNegociation(SCRAM-SHA-1, 03d48de8 ()...". – Joseph Hui Mar 28 '23 at 09:02
  • I have a testing environment which has a standalone MongoDB and I never see such errors. – Joseph Hui Mar 28 '23 at 09:02
  • "I believe the VPN is working fine" believing and be sure are not the same ... And the fact that it work one env without VPN go along that. – cchantep Mar 28 '23 at 12:47
  • I have another new application that reuse the same database related code of the problematic system. Their architectures are the same (Play 2.8 + PSA Replication + VPN), except the new one is running MongoDB v6 and the problematic one is running MongoDB v4. Also, both systems run on the same VPN network. So, I cannot think of any problem with the VPN. Could this be a problem with MongoDB v4? There is a hardware limitation and I cannot upgrade the MongoDB to newer version at the moment. I will see try upgrade the MongoDB at a later time. – Joseph Hui Mar 29 '23 at 02:11
  • The driver supports both version. MongoDB 6 itself may be better accross VPN (to advertise proper hosts), or maybe one cluster is well configured and not the other. In this kind of context, a lot of component can cause issues. – cchantep Mar 30 '23 at 15:02

0 Answers0