0

So basically I have a couple of different Glue jobs (all created from Terraform, but with different workspace for testing purpose), the Glue driver scripts are a little bit different, and they are stored in S3 bucket, then pointed to the targeted Glue job to use.

Strange thing is, I want to update the drive script for one of the job and test it, so I edit it in the AWS console, then when I check the glue script for other jobs, the driver scripts all gets updated automatically, I'm confused, why is this? I thought if I only modify Glue driver script for a specific job in the console, it shouldn't affect other jobs. Does anyone know what's going one here?

wawawa
  • 2,835
  • 6
  • 44
  • 105
  • your Glue driver is pointed to different S3 paths or to same S3 path ? – Prabhakar Reddy Aug 15 '21 at 02:25
  • @PrabhakarReddy Different S3 path, because I have different s3 buckets for different Terraform workspace (e.g: Glue job name: `xxxx-dev` uses S3 bucket `xxx-dev`, Glue job name `xxxx-test` uses s3 bucket `xxx-test`) – wawawa Aug 15 '21 at 15:55
  • In that case single update should not reflect in all scripts.Cross verify by going to each glue job manually and check the location. Once the script is modified check other scripts last modified time in S3 – Prabhakar Reddy Aug 16 '21 at 01:16

0 Answers0