I'm using distcp to move files between two hadoop clusters. How can I check the rate at which the data is moving between any two clusters?
Asked
Active
Viewed 583 times
2
-
1I know you are not looking for that (that's why I add this as a comment), but a dummy solution would be to run the distcp command using the `time` command, and then divide the data size by the returned time. (https://linux.die.net/man/1/time) – vefthym Mar 08 '17 at 08:58
-
check this link might yo will get some clarity https://developer.ibm.com/hadoop/2016/02/05/fast-can-data-transferred-hadoop-clusters-using-distcp/ – Hari Singh Mar 08 '17 at 10:51