I’m searching for some information about if is possible configure one scheduler processs of airflow per dag folder. I'm using separated dag folders depending on the business subject, and I need to know if is possible configure one scheduler process per which one of these folders. did someone knows if it is possible?
Asked
Active
Viewed 43 times
0
-
not aware of any such feature. why do you need such setup though? scheduler is pretty lightweight, so you need not have separate ones. can you share additional context. – Pbd Apr 20 '22 at 17:35
-
@Pbd My goal is achieve high availability. In my context I have many flows to different business subjects. I have sometimes to restart de scheduler to apply some configuration for example. But I don't want to cause downtime in the scheduler that prevents me from scheduling other tasks unrelated to the issue that caused the need to restart. Do you understand my need? – Bruno Apr 20 '22 at 17:44