1

I have a requirement to write Yarn application logs from EMR to different source other than S3 .. Can you please lep me where does applications logs get saved in EMR master instance

Manoj4068
  • 123
  • 1
  • 7
  • 14

2 Answers2

4

If the application is submitted to the emr as a step then the logs will reside in:

/var/log/hadoop/steps/<<step-id>>/<<log-file>>

most logs for emr can be found under the /var/logs directory in the master node

you could also use the yarn cli to get the application logs and redirect the returned log stream to a file to do whatever you want with.

yarn logs -applicationId <<application_id>> > application_log_file.log
randal25
  • 1,290
  • 13
  • 10
3

Yarn logs are found at /var/log/hadoop-yarn/, and yarn container logs are found at /var/log/hadoop-yarn/container

Links:

Community
  • 1
  • 1
N_C
  • 952
  • 8
  • 17