2

Guys!

val kafkaStream = KafkaUtils.createDirectStream[String, String, StringDecoder, StringDecoder](ssc, kafkaParams, topicSet)
kafkaStream.map(_._2).foreachRDD(rdd => rdd.foreach(println))

It worked ,it print kafka message.But when i run like this:

val dStreamsFilterByTableName = kafkaStream.transform(rdd => rdd.map(_._2).map(line => JSON.parse(line).asInstanceOf[JSONObject])
  .filter(json => OtterKafkaParser.filterJsonWithTableAndDB(json, tableName, dbName)))

dStreamsFilterByTableName.foreachRDD(rdd =>
  rdd.foreach(println))

It did not work.When i debugged in,there were data in dStreamsFilterByTableName,but did not print.

Why did it happen?

Thanks very much for your advice.

0 Answers0