0

Like we used to do with rdbms sql scripts. I wanted to do a similar thing with my dynamodb table.

Currently its very difficult to track changes from environment to environment(dev - qa -prod). We are directly making changes via the console.

What I want to do is, Keep the table data/json in the git version control and whenever any dev makes a change, we should be able to just run a script that will be able migrate the respective changes to on the dynamodb table eg. update/create/delete the tables, add/remove/update the records.

But I am not able to find a proper way/guide to achieve this currently. I am using javascript/nodejs as our base language.

Any help regarding this scenario will be appriciable.

Thanks

ref : https://forums.aws.amazon.com/thread.jspa?threadID=342538

Swapnil Mhaske
  • 365
  • 7
  • 21

1 Answers1

1

As far as I can tell you described to separate issues:

  1. Changing the tables "structure"
  2. Updating records after the update

Before I go into my answer, remember that DynamoDB is a NoSQL database and your previous RDBMS was a relational database. Operational tasks can differ very much for both types of databases.

1. Automating changes to the "structure" of the table

For this you can check out Infrastructure as Code tools like Terraform, CloudFormation or Pulumi.

But since DynamoDB is a NoSQL database, you only can do a few things like setting your hash and sort key etc and defining indices. Adding "fields" to the DynamoDB is not done with those tools, because except for the hash and sort key, there are no fields. Everything else does not follow a explicit (sql) schema.

2. Updating records after an update

If you do not have a lot of records, you could write yourself a simple tool or script to do the relevant work using the AWS SDK and run that during your CI/CD pipeline. A simple approach would be to have a "migrations" folder and if there is a file in it, the pipeline will execute it. So after the migration is done, just remove the file again. Not great, but pragmatic.

If you have a lot of records this won't work that great anymore, at least if you want to have a downtime-less deployment. In that case you will have update your software to be able to work with the old and new versions of the records structure, while you gradually update all records in the background (using a script etc.). Once all the records are updated, you can remove the code paths that handle the old structure.

Jens
  • 20,533
  • 11
  • 60
  • 86