Like AWS Kinesis Data Analytics SQL(legacy), Is it possible for the flink application on KDA to consume as firehose input stream ?
Asked
Active
Viewed 189 times
1
-
Questions that are phrased in a way that can be answered with "yes" or "no" are unlikely to get great answers. – David Buck Sep 07 '22 at 07:37
-
Did you ever find out the answer to this? – ironchicken Jun 15 '23 at 23:35
1 Answers
1
The documentation for the Kinesis Data Firehose SQL Connector has this to say:
The current implementation for the Kinesis Data Firehose SQL connector only supports Kinesis Data Firehose backed sinks and doesn’t provide an implementation for source queries. Queries similar to:
SELECT * FROM FirehoseTable;should result in an error similar to
Connector firehose can only be used as a sink. It cannot be used as a source.
This means that the connector that's included in Flink doesn't support using Firehose as a source.
In addition, Firehose is not really intended for streaming output. It's intended for loading the data from a stream input into other data sources such as S3, Elasticsearch or Redshift. So I guess that implies that a Firehose wouldn't be suitable as a Flink source.
ironchicken
- 754
- 6
- 19