Airflow error Command exited with return code 127

Question

I have a Ubuntu Server with airflow installed, I have a basic dag deployed there, the problem appear when I rebooted in the server and when I tried again to run the dag, airflow is showing this error:

[2022-11-29, 00:01:54 UTC] {subprocess.py:75} INFO - Running command: ['/usr/bin/bash', '-c', 'python /opt/***/dags/crypto/create_request.py 3000000 4000000']
[2022-11-29, 00:01:54 UTC] {subprocess.py:86} INFO - Output:
[2022-11-29, 00:01:54 UTC] {subprocess.py:93} INFO - /usr/bin/bash: line 1: python: command not found
[2022-11-29, 00:01:54 UTC] {subprocess.py:97} INFO - Command exited with return code 127
[2022-11-29, 00:01:54 UTC] {taskinstance.py:1851} ERROR - Task failed with exception
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/airflow/operators/bash.py", line 196, in execute
    raise AirflowException(
airflow.exceptions.AirflowException: Bash command failed. The command returned a non-zero exit code 127.

But before the reboot the dag was being executing normally.

The code is:

from airflow import DAG
from airflow.operators.bash import BashOperator
from airflow.utils.dates import days_ago
default_args = {
    "owner": "airflow",
    "depends_on_past": False,
    "email": ["pumasemj@hotmail.com"],
    'params': {
        "start_block": "3000000",
        "end_block": "4000000"
    }
}

with DAG("dag_infura", default_args=default_args, description='Dag for infura ingestion', schedule_interval=None, start_date=days_ago(2), tags=['dag_infura']) as dag:
    print('executing...')
    comd = "python /opt/airflow/dags/crypto/create_request.py {{params.start_block}} {{params.end_block}}"
    t1 = BashOperator(task_id='ingest', dag=dag,depends_on_past=False, bash_command=comd)

something I figure out is that before there was this transformation when executing the bash operator:

[2022-11-10, 00:04:50 UTC] {dagbag.py:525} INFO - Filling up the DagBag from /home/ubuntu/***/dags/pipeline-***/dag_crypto_infura.py
[2022-11-10, 00:04:50 UTC] {subprocess.py:75} INFO - Running command: ['/usr/bin/bash', '-c', 'python3 /home/ubuntu/***/dags/pipeline-***/create_request.py 3000000 4000000']

and now when it is failing:

INFO - Filling up the DagBag from /home/ubuntu/***/dags/crypto/dag_crypto_infura.py
[2022-11-28, 19:34:42 UTC] {subprocess.py:75} INFO - Running command: ['/usr/bin/bash', '-c', 'python /***/dags/crypto/create_request.py 3000000 4000000']
Running command: ['/usr/bin/bash', '-c', 'python /opt/***/dags/crypto/create_request.py 3000000 4000000']

Here is the key, but I don't know why now is changing in the execution of the command now, Any advice?

Regards

EDit: it's work when I change the command to:

 comd = "python3 /home/ubuntu/***/dags/pipeline-***/create_request.py  {{params.start_block}} {{params.end_block}}"

The problem now is that I develop from docker in windows and deploy to ubuntu without docker any advice to deal whith this situation? I don't want to manually replace the path every time I deploy to ubuntu

Airflow error Command exited with return code 127

0 Answers0