I'm revising one of our company's custom operator in a way so that it logs its activity to a database. As part of that, I generate a unique number for use in initially populating the table row before the task runs, and then use that same number to update the row with the results of the task run.
I leaned towards using callbacks to handle this logging functionality (it isn't hard to pass my own functions to the super constructor). The on_execute_callback function would generate the unique number and insert to the logging table, and either on_success_callback or on_failure_callback would update that row using the unique number.
The only problem is that I'm not sure how to pass the value generated in on_execute_callback to execute or any other callback. The params object passed within the functions' context parameter is specific to the task, not the task instance. I've looked around in the airflow.models.taskinstance module, but there isn't anything obvious about storing user data.
Is there a method I'm missing here for passing task-instance-specific data throughout the various methods? Is there a better way I should be doing this instead?
