How do we stream a data pipeline for data transfer from snowflake to kafka?

Question

I want to get the transformed data from snowflake to my postgres database. To achieve this I want to move the data from snowflake to kafka and then from kafka to postgres. Not able to figure out how to move data from snowflake to kafka.

I understand the following: You want to unload data from Snowflake to AWS/Azure/GCP and from there Kafka has to consume it? Right? — Marcel, Sep 30 '20 at 08:15

score 0 · Answer 1 · answered May 26 '21 at 15:58

You can use NiFi to get data from Snowflake and send it to a kafka topic
you can use a three party like Fivetran
You can use copy into to S3 or cloud storage and use a listener to get the new events https://docs.snowflake.com/en/user-guide/data-unload-s3.html

How do we stream a data pipeline for data transfer from snowflake to kafka?

1 Answers1