How to enter password during runtime in Airflow

Question

I have a script (RegisterKerbTicketAndOAuth.sh file) in remote cluster,

I am connecting to remote cluster from Airflow using ssh operator and logged in successfully which is successfull

Now the problem is, when I run the sh file like this

sh RegisterKerbTicketAndOAuth.sh  userID@azureprd.onmicrosoft.com

The above command will prompt for password twice at run time like this:

Enter password for this1:: XXXXXX
Enter password for this2:: XXXXXX

How do I pass my at run time in airflow

kinitCommandScript = """ 
sh RegisterKerbTicketAndOAuth.sh  userID@azureprd.onmicrosoft.com
"""

My Task Looks like this

kinitAuthentication = SSHOperator(
    ssh_hook = ConnectingToSSH, #SSHconnectionDetails
    task_id='GenerateTokenToRunDownstreamTasks',
    command= kinitCommandScript,
    #params={'my_param':NONFEDPASSWORD},
    dag=dag)

The above commented "#params" didnt worked. Can anyone suggest how to pass my password at run time for my shell script in airflow.

Note: Tried with BashOperator and its the same as above

score 0 · Answer 1 · answered Oct 18 '22 at 18:19

This is a question of how to run a shell script from some automated process, in general. The answer depends entirely on how your script is written and what commands/tools it calls. A common suggestion would be to use an environment variable to store the password. The SSHOperator supports this through its environment parameter. What environment variable you should use, or in fact whether this approach will work at all, 100% depends on what your script is doing.

How to enter password during runtime in Airflow

1 Answers1