I'm having problems having Python clients connecting to multiple servers that I set up using voldemort. I'm using Python 2.6.1 and Voldemort 0.81. Specifically, if I have a two node cluster, I don't seem to have any problems connecting to the first server, but can have large problems connecting to the second server. Here are the config settings that I'm using (in case this is causing some of the problems):
cluster.xml:
<cluster>
<name> my_cluster </name>
<server>
<id>0</id>
<host>1.2.3.4</host>
<http-port>8000</http-port>
<socket-port>6666</socket-port>
<admin-port>6667</admin-port>
<partitions>0,1,2,3</partitions>
</server>
<server>
<id>1</id>
<host>1.2.3.5</host>
<http-port>8000</http-port>
<socket-port>6666</socket-port>
<admin-port>6667</admin-port>
<partitions>4,5,6,7</partitions>
</server>
</cluster>
stores.xml:
<stores>
<store>
<name>multiple_nodes</name>
<replication-factor>2</replication-factor>
<preferred-reads>2</preferred-reads>
<required-reads>2</required-reads>
<preferred-writes>2</preferred-writes>
<required-writes>2</required-writes>
<persistence>bdb</persistence>
<routing>client</routing>
<key-serializer>
<type>string</type>
</key-serializer>
<value-serializer>
<type>string</type>
</value-serializer>
</store>
</stores>
server.properties:
node.id = 0 #(or 1)
max.threads = 100
http.enable = true
socket.enable = true
bdb.write.transactions = true
bdb.flush.transactions = true
enable.nio.connector = true
request.format = vp3
store.configs = voldemort.store.bdb.BdbStorageConfiguration, voldemort.store.readonly.ReadOnlyStoreageConfiguration
I've also had this problem with the values being json encoded and with the persistence just being in memory, so I don't believe the problems lie there. I've tried this with the two machines being in the same rack (communicating between them is trivial), and in separate racks (still should be absuredly fast, but not directly connected). Here's what I'm getting in Python:
>>>import voldemort
>>>client0 = voldemort.StoreClient('multiple_nodes', [('0', 6666)])
#That works just fine
>>>client1 = voldemort.StoreClient('multiple_nodes', [('1', 6666)])
WARNING:root:Metadata bootstrap from 1:6666 failed: Invalid Argument.
line 278 in _bootstrap_metadata #(I'm omitting a few other lines here)
voldemort.client.VoldemortException:'All bootstrap attempts failed'
I have this same result if I'm on the machine where server '0' is running and if I'm on the machine where server '1' is running. I've even tried connecting to both servers from the one client ala:
>>>client = voldemort.StoreClient('multiple_nodes', [('0', 6666), ('1', 6666)])
This occassionally works (although, it might crap out in the middle of testing), but occassionally fails with a warning similar to "WARNING:root:Metadata bootstrap from 1:6666 failed: Invalid Argument" while still giving me a client object back.
Any suggestions for how to fix this problem? I generally like the single node cluster performance I've seen, and would like to expand out to multiple nodes.