1

I am a newbie to NIFI, my use case is to read from a port and write to hdfs in parquet format, my research says there is something called KiteSDK, with which I can save as Parquet format.Am I right ?.Please advice.Any examples would help.

Community
  • 1
  • 1
Bill
  • 363
  • 3
  • 14

2 Answers2

0

There are certainly processors available in Apache NiFi to help you down the path. ListenTCP can capture data from a socket. You can use some of the transform processors potentially, depending on your type of data, to covert it. Not sure if Parquet as an output format is directly supported. And of course delivery to HDFS is supported. For missing pieces you can create a JIRA in the apache community or file a contribution or write an in-line script using ExecuteScript perhaps. Lots of options.

Thanks Joe

Joe Witt
  • 2,152
  • 10
  • 9
0

You can use the PutParquet processor of NiFi to do so. (Type PutParquet in "the filter input field": hyperlink under Processors.)

greybeard
  • 2,249
  • 8
  • 30
  • 66
Carrick
  • 23
  • 1
  • 4