I am trying to connect to a MongoDB replicaset with PyMongo and to manually balance the reading load with ReadPreference
.
The problem is that whatever I try with MongoClient
, it always reads from PRIMARY
.
I am using Python 2.7.6 and PyMongo 2.6.3 with mongodb-10gen
2.4.14 (all for legacy reasons) on Linux Mint 17.2.
My connection sequence looks like this (without the print, and with massive request on some collection of some database from the connection):
>>> from pymongo import MongoClient, ReadPreference
>>> HOST = "10.0.0.51"
>>> PORT = 49029
>>> print MongoClient(host=HOST, port=PORT, replicaset="rs02", readPreference=ReadPreference.SECONDARY)
MongoClient([u'XXXX-MNGO03664:49029', u'XXXX-MNGO03663:49029'])
This one would be the right way to go with PyMongo > 3, from what I have read. Unfortunately I am stuck with PyMongo 2.6.3, and I can only assume that this is why it doesn't read from secondary.
After a bit of digging, I found about ReplicaSetConnection
(deprecated since PyMongo 2.4) and MongoReplicaSetClient
(see e.g. pymongo replication secondary readreference not work and pymongo: Advantage of using MongoReplicaSetClient?), but it also doesn't seem to work for me, for different reasons though.
>>> print MongoReplicaSetClient(host=HOST, port=PORT, replicaset="rs02", readPreference=ReadPreference.SECONDARY)
MongoReplicaSetClient([])
The client doesn't seem to be able to see the members of the replicaset… And of course when I start reading from this connection, it doesn't work.
>>> myConn = MongoReplicaSetClient(host=HOST, port=PORT, replicaset="rs02", readPreference=ReadPreference.SECONDARY)
>>> print myConn.someDB.someCollection.count()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/pymongo/collection.py", line 759, in count
return self.find().count()
File "/usr/local/lib/python2.7/dist-packages/pymongo/cursor.py", line 640, in count
**command)
File "/usr/local/lib/python2.7/dist-packages/pymongo/database.py", line 391, in command
result = self["$cmd"].find_one(command, **extra_opts)
File "/usr/local/lib/python2.7/dist-packages/pymongo/collection.py", line 604, in find_one
for result in self.find(spec_or_id, *args, **kwargs).limit(-1):
File "/usr/local/lib/python2.7/dist-packages/pymongo/cursor.py", line 904, in next
if len(self.__data) or self._refresh():
File "/usr/local/lib/python2.7/dist-packages/pymongo/cursor.py", line 848, in _refresh
self.__uuid_subtype))
File "/usr/local/lib/python2.7/dist-packages/pymongo/cursor.py", line 782, in __send_message
res = client._send_message_with_response(message, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/pymongo/mongo_replica_set_client.py", line 1631, in _send_message_with_response
raise AutoReconnect(msg, errors)
pymongo.errors.AutoReconnect: No replica set secondary available for query with ReadPreference SECONDARY
Note that the same error appears when I try to read with ReadPreference.PRIMARY
.
The weird thing here is that if I change the name of the replicaset to connect to, the client spots that it doesn't exist :
>>> print MongoReplicaSetClient(host=HOST, port=PORT, replicaset="rs42", readPreference=ReadPreference.SECONDARY)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/pymongo/mongo_replica_set_client.py", line 742, in __init__
self.refresh()
File "/usr/local/lib/python2.7/dist-packages/pymongo/mongo_replica_set_client.py", line 1135, in refresh
% (host, port, self.__name))
pymongo.errors.ConfigurationError: 10.0.0.51:49029 is not a member of replica set rs42
So I assume that in normal cases, it has a way to see that there is a replicaset here, and who are its members.