0

I am performing distcp operation to copy the files from one directory to another in same cluster. It is a Kerberized Cloudera Hadoop Cluster.

Command I ran:

hadoop distcp -overwrite hdfs://nameservice/hdfs_path_source hdfs://nameservice/hdfs_path_target

I get below error:

java.io.IOException: Can't get Master Kerberos principal for use as renewer
        at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:132)
        at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:100)
        at org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:80)
        at org.apache.hadoop.tools.SimpleCopyListing.validatePaths(SimpleCopyListing.java:199)
        at org.apache.hadoop.tools.CopyListing.buildListing(CopyListing.java:85)
        at org.apache.hadoop.tools.GlobbedCopyListing.doBuildListing(GlobbedCopyListing.java:89)
        at org.apache.hadoop.tools.CopyListing.buildListing(CopyListing.java:86)
        at org.apache.hadoop.tools.DistCp.createInputFileListing(DistCp.java:368)
        at org.apache.hadoop.tools.DistCp.prepareFileListing(DistCp.java:96)
        at org.apache.hadoop.tools.DistCp.createAndSubmitJob(DistCp.java:205)
        at org.apache.hadoop.tools.DistCp.execute(DistCp.java:182)
        at org.apache.hadoop.tools.DistCp.run(DistCp.java:153)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
        at org.apache.hadoop.tools.DistCp.main(DistCp.java:432)```
Vijay
  • 123
  • 1
  • 11
  • Can you check Samson's comments in this SO post https://stackoverflow.com/questions/43769284/cant-get-master-kerberos-principal-for-use-as-renewer-for-talend-batch-jobs#56593666, plus whatever else google brings back for _Can't get Master Kerberos principal for use as renewer_ specifically with respect to setting up your env.vars (`HADOOP_HOME`, `HADOOP_CONF_DIR`, etc) – mazaneicha Apr 02 '21 at 20:32
  • This command worked for me. "hdfs dfs -cp -f /hdfs_path_source /hdfs_path_target" – Vijay Apr 08 '21 at 19:44

0 Answers0