1

I have created a data pipeline in Airflow which helps to update the tickets in Freshdesk whenever some values are change in the tickets during business hours. My logic for updatding the ticket is working fine, but I want to make the trigger the dags whenever a event is occured in Freshdesk interface so that I don't need to manually trigger the dags.

There are automation rules in FreshDesk which provides the external webhook in https://companyname.freshdesk.com/api/v2/tickets/{{ticket.id}} where {ticket.id} is essentail to know the id for which the tickets are updated from the data pipeline.

I need to know how to connect the external trigger such as webhook so that we can trigger the data-pipeline automatically in Airflow.

My Airflow Version 1.10

peerpressure
  • 390
  • 4
  • 19

2 Answers2

1

Airflow doesn't have a mechanism that allows triggering DAG based on webhooks from other services.

That use case might be covered by AIP-35 Add Signal Based Scheduling To Airflow but this is currently a draft idea of enhancement to Airflow.

To achieve your use case with current functionality you will need to other application/service to handle the webhook fired by Freshdesk and triggering a new DAG run in Airflow with RestAPI Trigger new DAG run endpoint

You need to be aware of frequency. If the frequency of creating new DAG runs is too high you should consider switching to a batch solution (which Airflow is more suitable for)

Elad Kalif
  • 14,110
  • 2
  • 17
  • 49
  • Thanks !! So I have to deploy another application(say Flask-based webserver) where we gather the events from the webhook and trigger the RESTAPI for the airflow. Currently, I'm having a standalone Airflow application. So essentially there is no way we can handle trigger Airflow's dags from webhooks via Freshdesk. – peerpressure Dec 17 '21 at 11:08
0

I am not sure about Freshdesk, but according to this it sounds as if you're able to hit an external API with its automation mechanism.

If that's the case, then I think this is what you're looking for. That's the documentation for the REST API reference of Apache Airflow.

Airflow allows you to manage it via web API, but you first need to enable it in the configuration. You should set the following configuration to use at least basic auth.

Keep in mind that back then that was an experimental API and newer versions changed the URL schema and the API a bit. It's much more complete nowadays.

EDG956
  • 797
  • 8
  • 23