1

I run 3 redis-server instances on my server, The instances have different config files and db files. e.g: redis1 set the db location as /var/lib/redis_1, and the redis2 set the db location as /var/lib/redis_2. I have the twemproxy config:

alpha:                                                                                                                                      
  listen: 0.0.0.0:9999
  redis: true
  hash: fnv1a_64
  distribution: ketama
  auto_eject_hosts: true
  server_retry_timeout: 2000
  server_failure_limit: 2
  servers:
   - 127.0.0.1:6381:1
   - 127.0.0.1:6382:1
   - 127.0.0.1:6383:1

I started the 3 redis instances, and set a data "name" as different values with redis-cli separately. e.g: I set the name as "6381" to the first server, and "6382" to the second one. I open the db files dump.rdb, and saw the 3 data values, then started twemproxy.

When I connect the twemproxy with redis-cli, and read the key "name", it always returns "6382". If I use different client to connect to it, the values are still the same. if I change the values directly in the Redis instabces, or do any other operations to twemproxy instance, I found the changes only updated to the second db file /var/lib/redis_2/dump.rdb.

Why doesn't twemproxy work with 3 redis-server instances? Is there any thing wrong with my config?

The Real Bill
  • 14,884
  • 8
  • 37
  • 39
varobj
  • 11
  • 1
  • 3

1 Answers1

7

I think you have a misunderstanding of how twemproxy works. I'll explain what is going on in your test and how you can validate twemproxy is connecting to each instance.

Twemproxy will use hashing to partition the key space and divide it amongst the three instances you have configured it for. THis means a key of 'name' will map to one and only one server. Twoemproxy does not use striping to place the data on all nodes, just one.

In your case 'name' maps to the instance running on port 6382. Twemproxy calculates the signature and uses the ketama distribution mechanism (per your config) to determine what instance should own 'name, then queries that instance when you do a get for 'name', returning the value you stored in it directly.

To validate twemproxy is connecting to each instance you could set a range of keys and then query each Redis instance directly. However, if querying directly you can use the info command to look at connection and operation stats.

  1. Connect directly to each one using redis-cli so the connection stays open.
  2. Start twemproxy
  3. Issue a series of gets using a wide variety of key names
  4. Check the connection stats for an increase in the number of connections

I don't recall at the moment if twemproxy connects to Redis instances when it starts. I think it does. If it does, you can do check connection stats after step 2. I would try this first.

In summary: it isn't your config it is your understanding of what should happen. If the above explanation doesn't clear it up for you let me know and I'll clarify/expand the answer.

Cheers

The Real Bill
  • 14,884
  • 8
  • 37
  • 39