1

when I ran command 

hadoop distcp -update hdfs://path/to/a/file.txt hdfs://path/to/b/

I got an Java IOException:

java.io.IOException: Mkdirs failed to create /some/.staging/directory

However, I don't want to use "/some/file/path" as a temporary staging directory, how can I change it to another directory I want?

konchy
  • 573
  • 5
  • 16
  • 1
    I think I've found it, it's "yarn.app.mapreduce.am.staging-dir" configuration in mapred-site.xml – konchy Mar 06 '20 at 10:55

1 Answers1

0

It's controlled via yarn.app.mapreduce.am.staging-dir (defaulting to /user in my install).

Ivan
  • 3,781
  • 16
  • 20