I want to use my function_two
multiple times with different values for the my_list
vairable.
I know I can use returned variables from one function into another using the xcom pull function. However, if I pull, I have to do it from within the python callable (for example in function_two
in this case). This means that I can only use the function with one my_list
value.
def function_one(**kwargs):
return my_list
def function_two(**kwargs):
my_list = task_instance.xcom_pull(task_ids='one')
# manipulating the list
get_one = PythonOperator(
task_id='one',
python_callable=get_one
)
get_two = PythonOperator(
task_id='two',
python_callable=get_two
)
get_three = PythonOperator(
task_id='three',
python_callable=get_two
op_args=.....
)
Instead, I want to re-use function_two
with different values of my_list
as parameters where I will be returning the lists from different other python callables.
Is there any way to pass specific task_instance
op_args in the PythonOperators such that I can call the same python callable in different operators but with different values for the my_list as the parameter?
For example, if I am returning another list here:
def new_function(**kwargs):
return new_list
I want to create a new PythonOperator that calls the same function_two
with this new_list
as the parameter.