4

I see Lettuce can do compression for Redis serialized objects: https://lettuce.io/core/release/reference/#codecs.compression

Any way to set this config within Spring Boot Data LettuceConnectionFactory or in some other bean? I've seen this question asked here as well: https://github.com/lettuce-io/lettuce-core/issues/633

I'd like to compress all serialized objects being sent to Redis to reduce network traffic between boxes.

Thanks

Sean
  • 111
  • 2
  • 8

2 Answers2

5

I ended up solving it the following way.

  1. Create a RedisTemplate Spring Bean. This allows us to set a custom serializer. 
    @Bean
    public RedisTemplate<Object, Object> redisTemplate(LettuceConnectionFactory connectionFactory) {
        RedisTemplate<Object, Object> template = new RedisTemplate<>();
        template.setConnectionFactory(connectionFactory);

        // Set a custom serializer that will compress/decompress data to/from redis
        RedisSerializerGzip serializerGzip = new RedisSerializerGzip();
        template.setValueSerializer(serializerGzip);
        template.setHashValueSerializer(serializerGzip);
        return template;
    }
  1. Create a custom serializer. I decided to extend JdkSerializationRedisSerializer since that's what Spring was using by default for Redis. I added a compress/decompress in each respected method and use the super class serialization code.
public class RedisSerializerGzip extends JdkSerializationRedisSerializer {

    @Override
    public Object deserialize(byte[] bytes) {
        return super.deserialize(decompress(bytes));
    }

    @Override
    public byte[] serialize(Object object) {
        return compress(super.serialize(object));
    }


    ////////////////////////
    // Helpers
    ////////////////////////
    private byte[] compress(byte[] content) {
        ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
        try (GZIPOutputStream gzipOutputStream = new GZIPOutputStream(byteArrayOutputStream)) {
            gzipOutputStream.write(content);
        } catch (IOException e) {
            throw new SerializationException("Unable to compress data", e);
        }
        return byteArrayOutputStream.toByteArray();
    }

    private byte[] decompress(byte[] contentBytes) {
        ByteArrayOutputStream out = new ByteArrayOutputStream();
        try {
            IOUtils.copy(new GZIPInputStream(new ByteArrayInputStream(contentBytes)), out);
        } catch (IOException e) {
            throw new SerializationException("Unable to decompress data", e);
        }
        return out.toByteArray();
    }

}
  1. Use spring dependency injection for the RedisTemplate Bean.
Sean
  • 111
  • 2
  • 8
  • Was there a specific reason why you used `IOUtils.copy(new GZIPInputStream(new ByteArrayInputStream(contentBytes)), out);` instead of just doing `GZIPOutputStream gzipOutputStream = new GZIPOutputStream(out); gzipOutputStream.write(contentBytes);` – PixelPusher Dec 12 '19 at 12:52
  • No specific reason. I think I just googled how to gzip data and used the first one on stack overflow I found. – Sean Dec 18 '19 at 02:42
0

Here is a sample of a Java Spring Boot Redis Cluster Data configuration.

It is an implementation with Redis Cluster and Redis Cache Manager.

  1. Snappy Compression
  2. Kryo Serialization
  3. Support ttl per cache key
  4. Gradle configuration
  5. spring-data-redis
  6. snappy-java
  7. kryo
  8. commons-codec

Link to github https://github.com/cboursinos/java-spring-redis-compression-snappy-kryo

Cbour
  • 49
  • 2
  • 9