0

I have a Dag A, It runs at a time let's say 10 Am, and typically completes within 15-20 mins, but sometimes it takes more time and due to some tables in the Database it goes into an endless running state, how can I know that if my DAG is completed within a given time frame and if not it should send email Alerts that it's not completed in this time and you need to check.

My thought process:

To build a parallel DAg or process within the same DAG and then write a python function in it which just checks the start time and match it with the Current time and then keeps subtracting it unless it reaches some fixed value lets say 10 mins and then shoots an email that it has not been completed.

Please correct me if I am wrong or what are the other ways to check it

ash
  • 1,065
  • 1
  • 5
  • 20
Aditya Verma
  • 201
  • 4
  • 14

1 Answers1

0

It sounds like you just need to define an SLA. You can find an example here.

joelrosenthal
  • 471
  • 4
  • 10