Snowflake: JSON data with dynamic columns

Question

Does Snowflake support realtime JSON data ingestion into tables using Kafka Connector with varying columns without predefining the schema? Also, is there any reference to the documentation/examples related to it.

Thanks in advance!

Regards, Kishore

Answer 1

Yes, Snowflake shines when ingesting JSON data without a predefined schema.

I just wrote about this:

• https://hoffa.medium.com/loading-reddit-comments-into-snowflake-44b9f2072a84

You'll notice that I'm reading JSON objects into a one column table - that column is a "variant" that will parse and optimize JSON objects on ingest.

Docs:

• https://docs.snowflake.com/en/user-guide/semistructured-intro.html

With Kafka:

• https://docs.snowflake.com/en/user-guide/kafka-connector.html