1

I am trying to figure out how to write a Spark Stream to a Phoenix table in the least convoluted way.

So far I have only found this solution: kafka-to-phoenix, which requires some deep ad-hoc engineering (to my noob eyes). I can tailor the linked solution to my needs without major issues, but I wonder if there are better solutions out there.

Basically, I arrive at this point in my code:

streamingData
    .writeStream
    .option("checkpointLocation", checkpointLocation)
    .trigger(Trigger.ProcessingTime(triggerInterval, TimeUnit.MILLISECONDS))
    .foreach(
        howToInsertToPhoenix?()
    )
    .start()
    .awaitTermination()

I have a running solution dumping the data to HBase, but it's not really less convoluted with the proposed Phoenix one, so it's not easily/promptly generalizable for similar use cases.

Many thanks in advance :)

Alex Ott
  • 80,552
  • 8
  • 87
  • 132
cyberZamp
  • 149
  • 1
  • 9

0 Answers0