0

We are running a migration of HDFS data from an HDP non-sercure cluster to CDP secure cluster, when I read the Cloudera documentation, they are mentioning "distcp" as a tool to handle the migration, but also they mention only from HDP secure cluster to CDP secure/non-secure cluster which is not my case.

I have few questions :

  • Should I secure the exiting cluster first and then use distcp ?
  • or is it okay if I use distcp without security checks ?
  • from you're experiences how can I handle such a situation ?

Thanks in advance

YasbyM
  • 1

1 Answers1

0

From my experience you will have to run the distcp from the CDP secure cluster, with a valid kerberos ticket, and with the following parameter :

ipc.client.fallback-to-simple-auth-allowed=true

Full example :

hadoop distcp \
   -D ipc.client.fallback-to-simple-auth-allowed=true \ 
   hdfs://<hdp_namenode>:8020/<dir> \ 
   hdfs://<cdp_namenode>:8020/<dir>
anisf
  • 1
  • 1