0

I am processing "DistCp" command to move few critical files form My Cluster1 to Cluster2. These critical files were residing with Blocksize 64MB, before. And now moved to Cluster2 [it got 128MB blocksize).

After the DistCp move, how does the does the critical files performance will increase with new blocksize in Cluster2..performance increase or decreases..???

1 Answers1

1

It depends on your files. Hadoop files are supposed to be read sequentially and if your files are big(let's say Gbs or Tbs) it would increase performance if you increase your blocksize, because it would decrease the number of tasks to be performed. Copying with Distcp will not maintain the block properties of your file since block configurations varies from cluster to cluster.

Leonardo Neves
  • 133
  • 1
  • 10