5

I'm trying to come up with an answer for a question of mine involving recovering a file when Windows' APIs complain about a "CRC error". I know this is because of a bad block.

But my theory is:

  • If I disable S.M.A.R.T., and hence ECC, I should be able to read the "raw data", without ECC applied, thus having a block with "uncertain data" in the file's stream (instead of a blank block with ECC applied).
  • This data should not match its NTFS' CRC, while other file's data should match theirs.
  • I should be able to brute force the whole file's "correct data" by flipping bits one by one of the block with "uncertain data" (I suspect corruption affects a bit or two in a bad block, not a whole block, hence it falls within the recoverability threshold of a 32-bit checksum and brute force).

Now, the problem is, I have no idea what this CRC is, or where it is. I Googled a lot, but I can't find about it. I imagine it is an NTFS feature, and as far as I know, NTFS doesn't have a complete, official documentation, and really I don't like the idea of looking for this in NTFS-3G's C++ source code.

So: I want to make a small utility or script to recover a files in this kind of scenario. An answer that tells me where to get this CRC and what algorithm it uses will be accepted.

Also: I suspect the CRC is for allocation units, not whole files. Maybe it's in the MFT?

Community
  • 1
  • 1
Camilo Martin
  • 37,236
  • 20
  • 111
  • 154
  • 1
    The CRC error is being reported by the hard drive itself, not NTFS. NTFS does not maintain CRCs. – Raymond Chen Mar 12 '12 at 08:43
  • http://en.wikipedia.org/wiki/Cyclic_redundancy_check – Cat Plus Plus Mar 12 '12 at 08:45
  • @RaymondChen If that was the case, how would chkdsk work? It does not work if the partition is not formated as NTFS or FAT. – Camilo Martin Mar 13 '12 at 04:24
  • @CatPlusPlus I know what a CRC is, thanks. If you posted that link because you think I'm too optimistic regarding "flipping bits one by one until the CRC matches", I'm still sure one bit of error would be fixable this way. I know it's not as good as a hash for this purpose. – Camilo Martin Mar 13 '12 at 04:28
  • 1
    Chkdsk checks file system integrity and therefore can operate only on file systems it understands. The CRC is checked by the physical hard drive circuitry. Chkdsk checks for logical damage. The CRC checks for physical damage. – Raymond Chen Mar 13 '12 at 06:57
  • @RaymondChen So in order to check for bad sectors, chkdsk just reads the sectors and catches the errors from the hard drive? Are you positively sure NTFS does not store some CRC? (I thought that it stored one because windows complains about it; I'd expect something more general if it's just reporting that the hard drive could not read a sector, something like "bad block" instead of "cyclic redundancy check error"). – Camilo Martin Mar 13 '12 at 08:33
  • The hard drive reports a specific reason for the error (seek error, sector not found, CRC error, DMA error, read fault, general failure, etc.) and Windows just reports that error back to you. – Raymond Chen Mar 13 '12 at 12:10
  • If you are positively sure about NTFS not having any kind of file-related CRC, post it as an answer and I'll accept it. – Camilo Martin Mar 13 '12 at 14:00
  • The error detecting codes used by disk drives are much more sophisticated than a CRC, and are already used by the drive to correct for multiple bit errors. If the drive is reporting an error it's because the error codes aren't sufficient to correct the error. http://en.wikipedia.org/wiki/Hard_disk#Error_handling – Mark Ransom Mar 13 '12 at 15:16
  • @MarkRansom After some more looking around, I'm curious. Why didn't we ever see bad data coming from a hard drive and coming out as "good"? How it manages so well to avoid letting an error through? – Camilo Martin Mar 15 '12 at 07:29

1 Answers1

7

The CRC errors you are seeing are not coming from NTFS. They are coming from the device driver.

Raymond Chen
  • 44,448
  • 11
  • 96
  • 135
  • It's sad, really, because I was expecting to be able to get this CRC somehow, but I'm sure it's not possible to get it out of the hardware (through normal means). Also, I'm just going to bother you a little bit more: if the hard drive returns data after several read attempts (instead of returning an error), does that mean the data was successfully read? Because, for some blocks reading data a dozen times seems to do the trick (and since it was a JPEG I could be sure it worked), and maybe if I set it to read a block indefinitely until it doesn't report an error that could do (I guess). Thanks! – Camilo Martin Mar 13 '12 at 16:55
  • If the hard drive reports that the data was successfully read, then that's what gets reported. Whether you trust the hard drive is your decision. – Raymond Chen Mar 13 '12 at 17:04
  • By the way, I just wanted to comment I'm trying to read the data using GNU `ddrescue`, and it DOES successfully read some blocks after several attempts, and the files are ok. I know this is merely anecdotal, but I thought someone might want to try it. Also, I'm in the very slow process of using SpinRite on this drive. I just listened to a full song that was corrupted, and that wouldn't be read even after some hundered attempts. It's surprising, but it did restore it (also, I know the bad block was mid-song (not metadata), and the song's spectrum looks smooth - it DID recover it, I'm amazed). – Camilo Martin Mar 15 '12 at 07:05
  • My apologies for reviving this Q&A, but is that still the case with Server 2016 and Hyper-V? I have a vhdx file restored from a backup software that reproducibly has files in its ntfs volume that show me a crc error when I try to open them. Please advise. – deucalion Apr 17 '18 at 20:21