0

I am using MWAA. I have included pysftp in mwaa requirements.txt file as

pysftp==0.2.9

In the scheduler log groups of MWAA I can even see that it is getting downloaded.

Collecting pysftp==0.2.9
  Downloading pysftp-0.2.9.tar.gz (25 kB)

but when I am trying to import it in my dags, I am getting the following error

Broken DAG: [/usr/local/airflow/dags/hello.py] Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/local/airflow/dags/hello.py", line 2, in <module>
    import pysftp
ModuleNotFoundError: No module named 'pysftp'

What might be causing this issue and what can be done to solve it?

Vibhav
  • 181
  • 5
  • 11
  • Is this the first time you add dependencies ? For the first time you also need to specify the path to the file on the Amazon MWAA console – Elad Kalif Jul 06 '21 at 11:21
  • yes, I have added a file called requirements.txt to s3. I have set it as my requirements file while editing the mwaa environment. I am also able to see the logs where its installing the libraries. But I am not able to use this library. – Vibhav Jul 06 '21 at 11:37
  • 1
    Just because it was downloaded, doesn't mean it was properly installed. Do the logs say anything else? You're looking for a message like "Successfully installed pysftp-0.2.9" at the end of the requirements_install log. – Tomas Farias Jul 21 '21 at 14:46

0 Answers0