Exporting dynamodb table as csv data pipeline

Question

I am trying to export my DynamoDB data as a .CSV file to S3. I've used;

{
  "id" : "DynamoDBDataType",
  "type" : "CSV",
  "column" : [
    "Name GsaDynamoDBDataType",
    "Score INT",
    "DateOfBirth TIMESTAMP"
  ]
}

and associated it with the s3 bucket to write to;

{
  "directoryPath": "#{myOutputS3Loc}/#{format(@scheduledStartTime, 'YYYY-MM-dd-HH-mm-ss')}",
  "name": "S3BackupLocation",
  "id": "S3BackupLocation",
  "type": "S3DataNode",
  "dataFormat": {
    "ref": "DynamoDBDataType"
  }
}

Any ideas?

Answer 1

It has to be done in 2 phases where the data is first exported to S3 in DynamoDB's export format. It can then be converted to CSV using another job. Here is a sample pipeline https://s3.amazonaws.com/dpl-examples/DDB-to-CSV.json