Flink Kafka Consumer throws Null Pointer Exception when using DataStream key by

Question

I am using this example Flink CEP where I am separating out the data as I have created one application which is Sending application to Kafka & another application reading from Kafka... I generated the producer for class TemperatureWarning i.e. in Kafka,I was sending data related to TemperatureWarning Following is my code which is consuming data from Kafka...

StreamExecutionEnvironment env=StreamExecutionEnvironment.getExecutionEnvironment();
env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);
env.enableCheckpointing(5000);
Properties properties=new Properties();
properties.setProperty("bootstrap.servers", "PUBLICDNS:9092");
properties.setProperty("zookeeper.connect", "PUBLICDNS:2181");
properties.setProperty("group.id", "test");



DataStream<TemperatureWarning> dstream=env.addSource(new FlinkKafkaConsumer09<TemperatureWarning>("MonitoringEvent", new MonitoringEventSchema(), properties));

  Pattern<TemperatureWarning, ?> alertPattern = Pattern.<TemperatureWarning>begin("first")
          .next("second")
          .within(Time.seconds(20));


  PatternStream<TemperatureWarning> alertPatternStream = CEP.pattern(
          dstream.keyBy("rackID"),
          alertPattern);


  DataStream<TemperatureAlert> alerts = alertPatternStream.flatSelect(
      (Map<String, TemperatureWarning> pattern, Collector<TemperatureAlert> out) -> {
          TemperatureWarning first = pattern.get("first");
          TemperatureWarning second = pattern.get("second");

          if (first.getAverageTemperature() < second.getAverageTemperature()) {
              out.collect(new TemperatureAlert(second.getRackID(),second.getAverageTemperature(),second.getTimeStamp()));
          }
      });
dstream.print();
alerts.print();

env.execute("Flink Kafka Consumer");

But when I execute this application,it throws following Exception:

Exception in thread "main" java.lang.NullPointerException
at org.apache.flink.api.common.operators.Keys$ExpressionKeys.<init>(Keys.java:329)
at org.apache.flink.streaming.api.datastream.DataStream.keyBy(DataStream.java:274)
at com.yash.consumer.KafkaFlinkConsumer.main(KafkaFlinkConsumer.java:49)

Following is my class TemperatureWarning :

public class TemperatureWarning {

private int rackID;
private double averageTemperature;
private long timeStamp;

public TemperatureWarning(int rackID, double averageTemperature,long timeStamp) {
    this.rackID = rackID;
    this.averageTemperature = averageTemperature;
    this.timeStamp=timeStamp;
}

public TemperatureWarning() {
    this(-1, -1,-1);
}

public int getRackID() {
    return rackID;
}

public void setRackID(int rackID) {
    this.rackID = rackID;
}

public double getAverageTemperature() {
    return averageTemperature;
}

public void setAverageTemperature(double averageTemperature) {
    this.averageTemperature = averageTemperature;
}


public long getTimeStamp() {
    return timeStamp;
}

public void setTimeStamp(long timeStamp) {
    this.timeStamp = timeStamp;
}

@Override
public boolean equals(Object obj) {
    if (obj instanceof TemperatureWarning) {
        TemperatureWarning other = (TemperatureWarning) obj;

        return rackID == other.rackID && averageTemperature == other.averageTemperature;
    } else {
        return false;
    }
}

@Override
public int hashCode() {
    return 41 * rackID + Double.hashCode(averageTemperature);
}

@Override
public String toString() {
   //return "TemperatureWarning(" + getRackID() + ", " + averageTemperature + ")";
    return "TemperatureWarning(" + getRackID() +","+averageTemperature + ") "+ "," + getTimeStamp(); 
}

}

Following is my class MonitoringEventSchema :

public class MonitoringEventSchema implements DeserializationSchema<TemperatureWarning>,SerializationSchema<TemperatureWarning>

{

@Override
public TypeInformation<TemperatureWarning> getProducedType() {
    // TODO Auto-generated method stub
    return null;
}

@Override
public byte[] serialize(TemperatureWarning element) {
    // TODO Auto-generated method stub
    return element.toString().getBytes();
}

@Override
public TemperatureWarning deserialize(byte[] message) throws IOException {
    // TODO Auto-generated method stub
    if(message!=null)
    {
    String str=new String(message,"UTF-8");
    String []val=str.split(",");
    TemperatureWarning warning=new TemperatureWarning(Integer.parseInt(val[0]),Double.parseDouble(val[1]),Long.parseLong(val[2]));
    return warning;
    }
    return null;
}

@Override
public boolean isEndOfStream(TemperatureWarning nextElement) {
    // TODO Auto-generated method stub
    return false;
}

}

Now what is required to do keyBy operation as I have mentioned the key which is required for stream to partition ?? What needs to be done here to solve this error ??

score 1 · Accepted Answer · answered Jun 02 '16 at 15:50

1

The problem is in this function:

@Override
public TypeInformation<TemperatureWarning> getProducedType() {
    // TODO Auto-generated method stub
    return null;
}

you cannot return null here.

answered Jun 02 '16 at 15:50

aljoscha

986
5
7

Thanks a lot... I used `return TypeExtractor.getForClass(TemperatureWarning.class)` that error got resolved. ...Just want to confirm that the way I have deserialized DataStream type class,is it correct way of deserializing the data ?? Can you please share me an example of custom KafkaConsumer example ?? – AKSHAY SHINGOTE Jun 03 '16 at 07:53
How can I resolve this error when KafkaFlinkConsumer is not able `java.lang.NullPointerException at com.yash.source.MonitoringEventSchema.deserialize(MonitoringEventSchema.java:74) at com.yash.source.MonitoringEventSchema.deserialize(MonitoringEventSchema.java:1) at org.apache.flink.streaming.util.serialization.KeyedDeserializationSchemaWrapper.deserialize(KeyedDeserializationSchemaWrapper.java:39) at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer09$ConsumerThread.run(FlinkKafkaConsumer09.java:466) ` – AKSHAY SHINGOTE Jun 03 '16 at 14:02

Flink Kafka Consumer throws Null Pointer Exception when using DataStream key by

1 Answers1