I have set up an AWS Data Pipeline of DynamoDB data into an S3DataNode, by using the DynamoDB->Export menu option that sets up the basic pipeline template.
I run that once a day, and it outputs into an S3 folder like "TableName/DATE/". I set that via the Directory Path option under the S3DataNode:
#{myOutputS3Loc}/#{myDDBTableName}/#{format(@scheduledStartTime, 'YYYY-MM-dd-HH-mm-ss')}
I would like to also have it output into an S3 folder like "TableName/LATEST" and have it always overwrite the contents there, so that there is a single canonical URL that always points to the latest backup JSON of a particular table.
I experimented with adding a CopyActivity and another S3DataNode, but it didn't seem to actually copy anything. Thanks for the help!