Airflow: How to setup log directory?

Question

I upload a dag file to the web page and when I click 'Graph View' -> ${my_dag} -> 'View Log', it shows:

*** Log file isn't local.
*** Fetching here: http://:8793/log/demo_dag/hello_task/2018-11-14T15:06:00
*** Failed to fetch log file from worker.

*** Reading remote logs...
*** Unsupported remote log location.

I have checked the airflow.cfg and find these config info:

worker_log_server_port = 8793
base_log_folder = /root/airflow/logs

My question is:

How to setup IP address for log service (Only port is setup)?
I have setup directory for log service, why does it still go to /log/.. ?

Any help is appreciated.

hey @joeb Thanks for your reply. The log file path is `/root/airflow/logs` (as in config file), but this path has full permissions (777). — user2894829, Nov 16 '18 at 02:43
What is the path to the file in your `/log` directory that you say has Airflow logging messages in it? — joebeeson, Nov 16 '18 at 14:03
@joeb thanks for your reply. The path is `/root/airflow/logs`, as shown in the config file. — user2894829, Nov 19 '18 at 05:21
In your second question "I have setup directory for log service, why does it still go to `/log/..`" -- What is the full path to the file in `/log/..` that you are seeing which contains Airflow logging? — joebeeson, Nov 19 '18 at 13:44

score 1 · Answer 1 · answered Nov 16 '18 at 01:46

1

This can happen when the task status was manually changed (likely through the "Mark Success" option) and the task never receives a hostname value on the record.

The webserver is attempting to reach out to a server, with no name, to get logs for a task that never ran.

PS: Be careful running processes as the root user.

answered Nov 16 '18 at 01:46

joebeeson

4,159
1
22
29

I don't think I changed the task status. Actually, all tasks keep running and never finished. I am trying to find reasons and logs are not available. – user2894829 Nov 16 '18 at 02:47
What is the status of the task that you're viewing? – joebeeson Nov 19 '18 at 13:45

score 1 · Answer 2 · answered Apr 06 '21 at 04:41

I've been getting this error, fix it by correcting the socket volume path:

WARNING - OSError while attempting to symlink the latest log directory

In windows the volume will go with a double bar like this:

volumes:
  - //var/run/docker.sock:/var/run/docker.sock

Bind to docker socket on Windows

Setting up Airflow to run with Docker Swarm’s orchestration

Airflow: How to setup log directory?

2 Answers2

Linked