1

I want to use my function_two multiple times with different values for the my_list vairable.

I know I can use returned variables from one function into another using the xcom pull function. However, if I pull, I have to do it from within the python callable (for example in function_two in this case). This means that I can only use the function with one my_list value.

def function_one(**kwargs):
   return my_list

def function_two(**kwargs):
   my_list = task_instance.xcom_pull(task_ids='one')
   # manipulating the list


get_one = PythonOperator(
        task_id='one',
        python_callable=get_one
)

get_two = PythonOperator(
        task_id='two',
        python_callable=get_two
)

get_three = PythonOperator(
        task_id='three',
        python_callable=get_two
        op_args=.....

)

Instead, I want to re-use function_two with different values of my_list as parameters where I will be returning the lists from different other python callables.

Is there any way to pass specific task_instance op_args in the PythonOperators such that I can call the same python callable in different operators but with different values for the my_list as the parameter?

For example, if I am returning another list here:

def new_function(**kwargs):
   return new_list

I want to create a new PythonOperator that calls the same function_two with this new_list as the parameter.

x89
  • 2,798
  • 5
  • 46
  • 110

0 Answers0