Questions tagged [amazon-kinesis-kpl]
52 questions
0
votes
3 answers
Load from S3 into Redshift using Kinesis
I receive my source files in S3, i need to load this data into redshift.
The source S3 files are populated via Kinesis Firehose.
Source :
via Kinesis
Webstreams -------------- > S3
Requirement :
via Kinesis
…
Srivignesh KN
- 452
- 8
- 22
0
votes
1 answer
Kinesis Agent Input - Log file created for each day
New Log file is created for each day with date appended in the name. For example "data_log-2017-05-14" log file is created for 14th May.
I need to set up Kinesis Agent to read data from the logs generated. But since each day a new log file is…
shruti
- 587
- 4
- 6
0
votes
1 answer
How to ensure that sent message with Kinesis Producer Library get delivered
I'm using KPL with AWS lambda (Java) for producing to Kinesis stream.
My code for adding messages is some thing like that:
ListenableFuture f = KP.addUserRecord(Stream, partitionKey,…
Jason4Ever
- 1,439
- 4
- 23
- 43
0
votes
1 answer
Kinesis Agent not sending records to Stream
I have built a Kinesis Firehose stream to push data into redshift and am trying to push data from an EC2 instance using kinesis agent.
Firehose Stream is able to parse the records but not identify the firehose streams am getting the following java…
Srivignesh KN
- 452
- 8
- 22
0
votes
1 answer
Kinesis Firehose load csv data into Redshift
I am using Kinesis firehose to process data into redshift and i am trying both Json and Csv formats.
The Json format works fine for me and data is getting loaded into redshift table.
COPY COMMAND USED FOR JSON :
COPY products_json FROM…
Srivignesh KN
- 452
- 8
- 22
0
votes
1 answer
Unable to read data from spark streaming connecting Kinesis
I have written below code to connect to kinesis from spark streaming but there is no data been received.
val kinesisStream = KinesisUtils.createStream(ssc, appName, streamName, endpointUrl, regionName, InitialPositionInStream.LATEST, batchInterval…
Mohan
- 3
- 1
- 4
0
votes
1 answer
Amazon Kinesis KPL SerializationException
I get
2016-03-21 10:17:19.915081] [0x00007f539b023740] [error] [retrier.cc:59] PutRecords failed: {"__type":"SerializationException"}
With the following Java stacktrace:
Caused by:…
Alexander Torstling
- 18,552
- 7
- 62
- 74