1

I am unable to delete corrupt files present in my HDFS.

when i launch : 

hdfs fsck -list-corruptfileblocks

i got some blocks corrupted : The filesystem under path '/' has 10 CORRUPT files

hdfs dfs -rm /path/to/filename.fileextension

rm: `/path/to/filename.fileextension': No such file or directory

I also tried

hdfs fsck / -delete 

  Under-replicated blocks:       0 (0.0 %)
     Mis-replicated blocks:         0 (0.0 %)
     Default replication factor:    3     
     Corrupt blocks:                0
     Missing replicas:              0 (0.0 %)   
    The filesystem under path '/' is HEALTHY

I've also restart both of namenodes and the bug still occurs

Can anyone please help me in this case ? Here is a similar case :

Thanks.

Il Sica
  • 11
  • 1
  • I don't understand. `hdfs fsck / -delete` returned 0 Corrupt blocks, which seems to be the thing you were trying to do in the first place. Correct me if I'm wrong – Abhinav Jul 30 '18 at 22:28
  • Exactly, it's confused for me too. The files doesn't exist anymore (that's why not found) but the hdfs fsck -list-corruptfileblocks still list them ! – Il Sica Jul 31 '18 at 08:34
  • Try to run `hdfs dfsadmin -metasave ` and check the output of this command. Does it show any corrupted blocks? – Abhinav Jul 31 '18 at 08:57
  • Nothing : Metasave: Blocks waiting for replication: 0 Mis-replicated blocks that have been postponed: Metasave: Blocks being replicated: 0 Metasave: Blocks 0 waiting deletion from 0 datanodes. Corrupt Blocks: – Il Sica Jul 31 '18 at 09:25
  • So no need to worry. Ignore the error. Your cluster is healthy. – Abhinav Jul 31 '18 at 14:22
  • I know it is, but I want to delete those warnings. Thanks for your help anyway – Il Sica Jul 31 '18 at 14:49

1 Answers1

1

maybe the corrupt files are in snapshot, and you should delete snapshot

https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsSnapshots.html

AbnerYH.Sun
  • 11
  • 1