I have some corrupted files on my Hadoop machine and I want to transfer them to another computer and see what is in them.
I tried to do hadoop fsck -copyToLocal /dir1/ /dir2/. It gives nothing. When I do hadoop fs -copyToLocal /dir1/ /dir2/ or hadoop dfs -copyToLocal /dir1/ /dir2/. It writes:
/opt/atsd/hadoop/bin/hadoop fs -copyToLocal /hbase/ /home/axibase/Documents/new/
15/04/15 07:12:32 WARN hdfs.DFSClient: Failed to connect to /192.168.1.211:50010, add to deadNodes and continuejava.io.IOException: Got error for OP_READ_BLOCK, self=/192.168.1.211:59175, remote=/192.168.1.211:50010, for file /hbase/atsd_d/06ea5db6b3cda82baa0d8af17cc36fed/r/24a1aa779e2c422db84369bfe2236003.edf12e41aa2aaa845eb092d661fb836e, for block 1601256999140141614_1170
15/04/15 07:12:32 INFO hdfs.DFSClient: Could not obtain block blk_1601256999140141614_1170 from any node: java.io.IOException: No live nodes contain current block. Will get new block locations from namenode and retry...
15/04/15 07:12:35 WARN hdfs.DFSClient: Failed to connect to /192.168.1.211:50010, add to deadNodes and continuejava.io.IOException: Got error for OP_READ_BLOCK, self=/192.168.1.211:59176, remote=/192.168.1.211:50010, for file /hbase/atsd_d/06ea5db6b3cda82baa0d8af17cc36fed/r/24a1aa779e2c422db84369bfe2236003.edf12e41aa2aaa845eb092d661fb836e, for block 1601256999140141614_1170
15/04/15 07:12:35 INFO hdfs.DFSClient: Could not obtain block blk_1601256999140141614_1170 from any node: java.io.IOException: No live nodes contain current block. Will get new block locations from namenode and retry...
15/04/15 07:12:38 WARN hdfs.DFSClient: Failed to connect to /192.168.1.211:50010, add to deadNodes and continuejava.io.IOException: Got error for OP_READ_BLOCK, self=/192.168.1.211:59177, remote=/192.168.1.211:50010, for file /hbase/atsd_d/06ea5db6b3cda82baa0d8af17cc36fed/r/24a1aa779e2c422db84369bfe2236003.edf12e41aa2aaa845eb092d661fb836e, for block 1601256999140141614_1170
15/04/15 07:12:38 INFO hdfs.DFSClient: Could not obtain block blk_1601256999140141614_1170 from any node: java.io.IOException: No live nodes contain current block. Will get new block locations from namenode and retry...
15/04/15 07:12:41 WARN hdfs.DFSClient: Failed to connect to /192.168.1.211:50010, add to deadNodes and continuejava.io.IOException: Got error for OP_READ_BLOCK, self=/192.168.1.211:59178, remote=/192.168.1.211:50010, for file /hbase/atsd_d/06ea5db6b3cda82baa0d8af17cc36fed/r/24a1aa779e2c422db84369bfe2236003.edf12e41aa2aaa845eb092d661fb836e, for block 1601256999140141614_1170
15/04/15 07:12:41 WARN hdfs.DFSClient: DFS Read: java.io.IOException: Could not obtain block: blk_1601256999140141614_1170 file=/hbase/atsd_d/06ea5db6b3cda82baa0d8af17cc36fed/r/24a1aa779e2c422db84369bfe2236003.edf12e41aa2aaa845eb092d661fb836e
at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.chooseDataNode(DFSClient.java:2269)
at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.blockSeekTo(DFSClient.java:2063)
at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:2224)
at java.io.DataInputStream.read(DataInputStream.java:100)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:68)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:47)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:87)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:341)
at org.apache.hadoop.fs.FsShell.copyToLocal(FsShell.java:248)
at org.apache.hadoop.fs.FsShell.copyToLocal(FsShell.java:272)
at org.apache.hadoop.fs.FsShell.copyToLocal(FsShell.java:272)
at org.apache.hadoop.fs.FsShell.copyToLocal(FsShell.java:272)
at org.apache.hadoop.fs.FsShell.copyToLocal(FsShell.java:272)
at org.apache.hadoop.fs.FsShell.copyToLocal(FsShell.java:199)
at org.apache.hadoop.fs.FsShell.run(FsShell.java:1769)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.fs.FsShell.main(FsShell.java:1895)
Also I tried to do hadoop fsck -move. It removes corrupted files, and I can't find them.
