1

In order to apply devops principles to data (ugh, dataops!), things like continuous deployment need to be considered.

Hence why tools like dbDeploy exist. However dbDeploy seems to have been orphaned and is not maintained any more. In the past i've used this tool again and again, but I don't see much support for it, and I'm not sure why?

So i'm wondering just what do people use to manage and version their schemas. In particular i'm looking for something that will work with Athena (But this has a jdbc driver, so in theory any jdbc compliant tool)

I know one answer may be to switch mindset, and use the AWS Glue crawlers instead. But do people actually do that? Or are the crawlers more for POC/Quick start situations? I'm pretty sure you'll always want to override decisions the crawler makes, so how can that be handled?

Codek
  • 5,114
  • 3
  • 24
  • 38
  • AWS Glue has many API functions, and not only based on crawlers. You can check the tables using these calls, such as: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/glue.html#Glue.Client.get_table – Guy Oct 18 '19 at 19:18
  • OK, all feels a bit homegrown - i'd assumed there would be something prebuilt for doing this! That is helpful though, I was thinking of getting such info from jdbc but the API may actually be easier. – Codek Oct 23 '19 at 15:31

0 Answers0