Topic partition appears as null when application is run on local Kafka instance

Question

I have this producer:

public class ProducerDemo {

    public static void main(String[] args) {
        
        String inputFile = "C:\\Users\\path\\to\\binary\\file";
        String bootstrapServers = "127.0.0.1:9092";
        
        try (
                InputStream inputStream = new FileInputStream(inputFile);
                OutputStream outputStream = new FileOutputStream(outputFile);
            ) {
     
                long fileSize = new File(inputFile).length();
     
                byte[] allBytes = new byte[(int) fileSize];
     
                inputStream.read(allBytes);
        
        // create producer properties
        Properties properties = new Properties();
        
        properties.setProperty(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers);
        properties.setProperty(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
        properties.setProperty(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, ByteArraySerializer.class.getName());
        properties.setProperty(ProducerConfig.INTERCEPTOR_CLASSES_CONFIG, "com.cme.pe.kms.EncryptionInterceptor");
        
        // create the producer
        KafkaProducer<String, byte[]> producer = new KafkaProducer<String, byte[]>(properties);
        
        // create a producer record
        ProducerRecord<String, byte[]> record = new ProducerRecord<String, byte[]>("foo", allBytes);
    
        // send asynchronously
        producer.send(record);
        
        // flush and close the producer
        //producer.flush();
        producer.close();
        
        } catch (IOException ex) {
            ex.printStackTrace();
        }

    }

I also have a Kafka Intereptor with this method:

public ProducerRecord<String, byte[]> onSend(final ProducerRecord<String, byte[]> record) {
    try {
        MasterKeyProvider<?> key_provider = keySet.keyProvider(record.topic());
        Toppar toppar = new Toppar(record.topic(), record.partition(), 0L, 0);
        inboundDecoders.get(record.topic()+record.partition()).decodeMsg(record.value(), toppar);
        byte[] buf = encrypt(key_provider, record.value());

        return new ProducerRecord<>(record.topic(), record.partition(), record.key(), buf);
    } catch (Exception e) {
        LOGGER.error("unable to encrypt message", e);
        return new ProducerRecord<>(record.topic(), record.partition(), record.key(), new byte[0]);
    }
}

When I run my application I get this error:

java.lang.NullPointerException at com.my.app.MyInterceptor.onSend(MyInterceptor.java:44) .... at com.my.app.ProducerDemo.main(ProducerDemo.java:50)

Line 44 is Toppar toppar = new Toppar(record.topic(), record.partition(), 0L, 0);. I think the issue is that record.partition() is null. I'm running Kafka as a single instance locally. I thought the partition would be 0. Am I seeing correct functionality or how do I resolve this?

You need to show the Toppar class. Passing null alone as a parameter doesn't throw an NPE — OneCricketeer, Mar 28 '21 at 15:08

score 0 · Answer 1 · answered Mar 11 '21 at 18:47

Your understanding that record.partition() is null is indeed correct.

You have not provided topic or partition during the creation of producer record. Not providing partition is still understandable but you should provide the topic to which the record needs to be sent to.

Kafka has it own custom logic to determine the partition incase you do not provide a partition in ProducerRecord (skipping details to keep this short). In case you are doing this as some POC, you can create producer record with partition=0 which will resolve your issue. It all boils down to what you are trying to achieve in your interceptor.

Topic partition appears as null when application is run on local Kafka instance

1 Answers1