1

In AWS Glue, I am executing a couple of ETL jobs using workflow, Now I want to inform business via email on the failure of any of the ETL jobs. I need help to get name of failed job and pass it to job which would trigger an email.

Pratik Mehta
  • 13
  • 1
  • 5

1 Answers1

6

Step 1: Create a topic in Amazon SNS, with Protocol as Email as well as Create subscription and confirm subscription

Step 2: Create new CloudWatch Events rule with following custom event pattern under Event Source section;

{
  "source": [
    "aws.glue"
  ],
  "detail-type": [
    "Glue Job Run Status"
  ],
  "detail": {
    "state": [
      "FAILED",
      "ERROR",
      "TIMEOUT"
    ]
  }
}

Step 3: For Cloud Watch Event Rule, under the Targets section, choose Add targets, and then change the default Lambda function to SNS topic and choose name of SNS topic created in Step 1. Finish creation of this CloudWatch event rule.

amitd
  • 1,497
  • 4
  • 11
  • AWS is moving the events management to EventBridge. – fernolimits Feb 07 '21 at 14:51
  • This answer is a little bit outdated. Foremost, events have been moved to EventBridge service. Also in the pattern, there should be "Glue Job State Change" detail type. Last but not least, instead of custom Lambda function, create a Step Function that sends event (from EventBridge) rule to SNS topic. – Pawel Aug 17 '22 at 08:38