2

I'm writing some benchmarks for a new project and I'm running into an issue where the lettuce(version 5.1.3) and I'm finding the code below which uses mget:

 @Override
public Set<V> getKeys(final Set<Long> keySet) {

    try {
        return asyncCommands.mget(keySet.toArray(new Long[0])).get(
                5, TimeUnit.SECONDS
        ).stream().map(keyValue -> keyValue.getValue()).collect(Collectors.toSet());
    }  catch (Exception e) {
        throw new RuntimeException(e);
    }

to be considerably slower(like over 100x slower) than using pipelining myself:

List<RedisFuture<V>> futures = new ArrayList<>(keySet.size());
    keySet.forEach(
            key -> futures.add(asyncCommands.get(key))
    );

    asyncCommands.flushCommands();

    LettuceFutures.awaitAll(5, TimeUnit.SECONDS,
        futures.toArray(new RedisFuture[0]));

    final Set<V> collect = futures.stream().map(
            future -> {
                try {
                    return future.get(1, TimeUnit.SECONDS);
                } catch (Exception e) {
                    log.warn("", e);
                    throw new RuntimeException();
                }
            }
    ).filter(
            Objects::nonNull
    ).collect(Collectors.toSet());
    return collect;

Both of these seem quite slow compared to what the redis server is reporting, but there could be other factors there. The javadocs say that the mget should use pipelining, so why is it so much slower than when I do the pipeline myself? What am I not doing right?

Edit: for the mget I have autoflushcommands enabled, for the pipelining it is disabled.

Update: Found out the culprit behind the slow performance is a slow codec, is there any way I can increase the overall throughput when the codec is slow?

user439407
  • 1,666
  • 2
  • 19
  • 40
  • How are you implementing the codec? – rainhacker May 08 '19 at 19:01
  • Data is serialized Protocol buffer data, so I am just calling the deserialize method provided by the Protocol Buffer java library and passing in a byte array that I got from the ByteBuffer, something along the lines of: `public MyProtobufClass decodeValue(final ByteBuffer bytes) { return MyProtobufClass.parseFrom(new ByteBufferBackedInputStream(bytes); }` – user439407 May 09 '19 at 08:27
  • I've seen Kryo performing better than protobufs for my case. If you have an option to try out a different code - I recommend giving Kryo a try. Redisson - a contender of lettuce has an implementation that you can refer: https://github.com/EsotericSoftware/kryo – rainhacker May 09 '19 at 20:03

0 Answers0