How to send JAVA objects to kafka using compression provided by KAFKA?

Question

So I am trying to send an object of around 5 MB to Kafka and I intend to use Kafka provided compression for it. I have not changed any default properties. My producer properties are:-

public static Producer<String, TestZip> createProducer() {
        Properties props = new Properties();
        props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, IKafkaConstants.KAFKA_BROKERS);
        props.put(ProducerConfig.CLIENT_ID_CONFIG, IKafkaConstants.CLIENT_ID);
        props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
        props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, TestZipSerializer1.class.getName());
        props.put(ProducerConfig.COMPRESSION_TYPE_CONFIG, "gzip");

        return new KafkaProducer<>(props);
    } 

Following is my custom serializer for the object

    public byte[] serialize(String arg0, TestZip o) {
              byte[] retVal = null;
              try {
                          ByteArrayOutputStream baos = new ByteArrayOutputStream();
                          ObjectOutputStream oos = new ObjectOutputStream(baos);
                          oos.writeObject(o);
                          oos.flush();
                          oos.close();
                          baos.close();
                          retVal = baos.toByteArray();
                          return retVal;
                      } catch (IOException e) {
                         System.out.println(e);
                      }

              return null;
        }
Ignore resource closing and other stuff here

When I try to send the object using the above property and serializer, I get an error saying

java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.RecordTooLargeException: The message is 5115700 bytes when serialized which is larger than the maximum request size you have configured with the max.request.size configuration 

This is exactly the size of the object I created and I don't understand why it is not getting compressed.

At the same time, if I change my custom serializer to use GZip compression as

public byte[] serialize(String arg0, TestZip o) {
          byte[] retVal = null;
          try {
                      ByteArrayOutputStream baos = new ByteArrayOutputStream();
                      GZIPOutputStream os = new GZIPOutputStream(baos);
                      ObjectOutputStream oos = new ObjectOutputStream(os);
                      oos.writeObject(o);
                      oos.flush();
                      oos.close();
                      baos.close();
                      retVal = baos.toByteArray();
                      return retVal;
                  } catch (IOException e) {
                     System.out.println(e);
                  }

          return null;
    }

Using this serializer, my object gets sent successfully whether or not I use the gzip property in the producer.

Could anyone tell me the correct way of using Kafka provided compression for sending a JAVA object as I do not want to compress in my custom serializer?