Use multithreading with redis , using jedis connection pool

Question

I have read that redis is single threaded per user/client.

Is it possible to use jedis connection pool to achieve multithreaded behaviour?

Basically different jedis client request would be using different connection obatined from jedis connection pool , to fire their commands to one redis server.

Since client requests are using different connections hence one redis should be server multiple requests parallely right ?

score 2 · Answer 1 · answered May 23 '18 at 08:26

Is it possible to use jedis connection pool to achieve multithreaded behaviour?

Though you use multiple connection or threading or processing to communicating with redis, These commands are all put on one queue, and redis will get one by one and execute one by one.And every single command is atomic. So this is still a single thread behaviour at the aspect of the redis.

Since client requests are using different connections hence one redis should be server multiple requests parallely right ?

Not always, the commands from different clients may arrive at redis at arbitrary order, so if these command has some data relation, dangerous things will happen.

consider this simple scenario, if redis has a counter S, if all client do the INCR command, then it is ok, because every command is executed atomicly, But if some client's code is like something that:

s = get S
t = s + 10
set S t

Then things would be wrong. And you should use the Multi/exec command to ensure that multi commands would be atomic no just only one command. More things about transactions in redis, you could refer this page. It is detailed.

Imaskar · Answer 2 · 2018-05-23T07:42:45.723

0

I answered similar question here, you may check it for details.

TL/DR: Redis itself will still be single threaded, but in the grand scheme of things time used by Redis to answer is order of magnitude less than time used by networking. You will get benefit from multithreading, yes.

EDIT: Along the request pipeline you will have 3 points where requests are processed sequentially. Its when the data packets are sent through a wire (you only have one wire, but the packets from different requests will be mixed) forward and backward, and when Redis serves your request. But Redis itself is no dumb either. Your requests will come from a network socket and will be pre-processed and written into client buffers. This part is processed in parallel. Then Redis's main loop will pick your commands from a queue, process and write response into client buffers. This part is done sequentially. Then responses from client buffers will be packed and sent through a wire back to you. This is also being done in parallel.

edited May 23 '18 at 07:42

answered May 23 '18 at 07:28

Imaskar

2,773
24
35

still can you answer if commands fired from 2 different connections from one client be served parallely by 1 redis server? If no then why is redis single threaded only on user level? – Manas Saxena May 23 '18 at 07:31
why is there only one wire while sending request to redis, when I have used multiple different connections using jedis connection pool? – Manas Saxena May 23 '18 at 07:49
Well, how many wires there are between your client and redis instances? Usually its one. You can actually multiplex it, but this is rarely useful. And if client and server are on the same physical machine, you most likely save on that part, depending on OS level drivers. – Imaskar May 23 '18 at 07:57
But don't get wrong impression. If you send a request A from thread 1 and at the (virtually) same time request B from thread 2, then it is possible that you will get response to A or B first. Threads won't be waiting for each other. – Imaskar May 23 '18 at 08:00
@ManasSaxena would you consider accepting one of the answers or something is still unclear for you? – Imaskar May 28 '18 at 10:23

score 0 · Answer 3 · answered May 23 '18 at 13:06

This works fine in my (web based) multi-threaded environment.

Make sure that RedisInterface instance be static in your class.

public class RedisInterface implements Closeable
{   
/*      
<dependency>
<groupId>redis.clients</groupId>
<artifactId>jedis</artifactId>
<version>2.9.0</version>
</dependency>   
*/

private JedisPool jedisPool =  null;

private synchronized void initializePool()
{
    if(jedisPool!=null) return;

    JedisPoolConfig poolConfig = new JedisPoolConfig();
    poolConfig.setMaxTotal(Integer.parseInt(AppConfig.REDIS_MAX_CONN)); //max conn count
    poolConfig.setMaxIdle(Integer.parseInt(AppConfig.REDIS_MAX_IDLE_CONN)); //max idle conn count
    poolConfig.setMaxWaitMillis(Long.parseLong(AppConfig.REDIS_MAX_WAIT_TIME)); // max wait time for new connection (before throwing exception)

    jedisPool = new JedisPool(poolConfig, 
            AppConfig.REDIS_IP, 
            Integer.parseInt(AppConfig.REDIS_PORT),
            Integer.parseInt(AppConfig.REDIS_CONN_TIMEOUT));

}

//not synchronized after testing thread safety of jedisPool.getResource()
protected Jedis getJedis()
{               
    if(jedisPool==null)
        initializePool();

      Jedis jedis = jedisPool.getResource();

      return jedis;     
}


public Long set(final byte[] key, final byte[] field, final byte[] value)
{
    Jedis redis = null; 
    Long ret =0L;

    try
    {
        redis = getJedis();

        ret = redis.hset(key, field, value);            
        redis.expire(key, Integer.parseInt(AppConfig.REDIS_EXPIRE_MIN)*60); //      
    }
    finally
    {
        if(redis!=null)         
            redis.close();
    }

    return ret;
}

public byte[] get(final byte[] key, final byte[] field) {

    Jedis redis = null ;
    byte[] valueBytes = null;

    try
    {
        redis = getJedis();

        valueBytes = redis.hget(key, field);
    }
    finally
    {       
        if(redis!=null)             
            redis.close();
    }

    return valueBytes;

}

@Override
public void close() throws IOException {

    if(jedisPool!=null)     
        jedisPool.close();      
}

}

Use multithreading with redis , using jedis connection pool

3 Answers3