3

I'm very very new to Apache-airflow, just started a course in udemy (this course) .

We have recieved YAML file and were asked to follow instructions in order to install airflow. I have been able to install airflow, I beleive, because I have gotten any error while doing the next steps:

  1. create a new file new_file.env (Visual Studio)
  2. add these lines inside new file and save it :
AIRFLOW_IMAGE_NAME=apache/airflow:2.3.0
AIRFLOW_UID=50000
  1. open the terminal and run
docker-compose up -d

Then when I tried to open localhost:8080 nothing was opened.

When I checked the containers, using

docker-compose ps

I have seen that some of the containers are not healthy.

docker logs materials_name_of_the_container #here I inserted every tome different container name

The most common error was "unable to configure handler "processor"

I have seen here post with similar problem that recommened to use pip to install Azure ,and I have tried to print this in the terminal:

pip install 'apache-airflow[azure_blob_storage,azure_data_lake,azure_cosmos,azure_container_instances]

#also tried with one more '

pip install 'apache-airflow[azure_blob_storage,azure_data_lake,azure_cosmos,azure_container_instances]'

I got an error that pip is not recognized: enter image description here

My goal is to complete the installation , and "on -the-way" to understand why these errors happened. I beleive the the YAML file is well (because it's an organized course), but no idea where or what happenned inside the contaienrs, and where these containers are located, so any "dummy" explaination for beginner is welcome.

Reut
  • 1,555
  • 4
  • 23
  • 55
  • does your docker container have python installed – Deepan Aug 08 '22 at 08:16
  • @Deepan I haven't done this actively . Is it something that comes within the YAML? – Reut Aug 08 '22 at 08:53
  • have you installed python in your windows machine and added python to `path` – Deepan Aug 08 '22 at 15:50
  • I have python on my widnows machine. Should I "copy" it to the folder? – Reut Aug 09 '22 at 06:33
  • https://airflow.apache.org/docs/apache-airflow/stable/start/docker.html follow instruction from here – Deepan Aug 09 '22 at 06:38
  • Noting the question you shared points to Azure because the question was on wasb which is Azure service. Azure has nothing to do with your actual problem of installing Airflow. – Elad Kalif Aug 10 '22 at 18:22

1 Answers1

2

In the end I could solve this issue , after reading this troubleshooting page. I was trying to configure the users&groups, then I have realized that for some reason, my user is not the administrator. I have changed it, uninstall everything (docker+visual studio) and then installed it again , and- yayyy! it worked and this error didn't appear.

so the steps I did: ~the foirst part of changing to administrator- I have done yesterday and I don't remember 100% how, I think it was this, if it's wrong please let me know...

  1. Enter the users&groups - that was not available in my windows 10, if I remember correct, I have pressed "windows" +"R" on the keyboard, and then I typed "netplwiz" and pressed enter. enter image description here
    2.double click on the username, and this tab was opened:

![enter image description here

  1. select administraor (the screenshow is after the fix so maybe it doesn't have the docker options)

4. restart the computer
5. uninstalled docker and visual studio
6. installed docker and visual studio again
7. docker-compose up -d ....
then after few minutes it worked.

Thanks for everyone who tried to help me :)

let me know if there is somethin g to improve in this answer.

Reut
  • 1,555
  • 4
  • 23
  • 55