2

some presentation declares that zfs has:

zfs can detect and correct silent data corruption.

e.g. from here http://www.eecis.udel.edu/~bmiller/DE-OSUG/ECECIS-ZFS.pdf

  1. But do you need to allocate some spare disk or zfs pool to do it mannually? or this is intrinsic by zfs?
  2. Does single disk zfs file system have this feature or you have to get RAIDZ?
c2h2
  • 768
  • 2
  • 9
  • 20
  • 2
    You should really unaccept Gregory's answer. It's plain wrong. This is a good example of why accepting the first answer very quickly is usually a bad idea. You should let it sit for a day or two and get community feedback. – MDMarra Apr 22 '12 at 11:01

2 Answers2

12

1. But do you need to allocate some spare disk or zfs pool to do it manually? or this is intrinsic by zfs?

The affected data need to be redundant for this to happen. This redundancy can be achieved without extra disks. Multiple disks doesn't imply redundancy either.

ZFS supports spare devices but they are here to replace other devices that are in failed state. They are not used for data redundancy.

2. Does single disk zfs file system have this feature or you have to get RAIDZ?

Whatever the pool configuration, corrupted data is always detected with ZFS unless you explicitly disable checksums, but that would be a very bad idea.

A single disk pools can recover a rotten block when it contains metadata. Blocks containing file data can only be recovered if the copies property is set to 2 or higher.

Multiple disk pools in a striped configuration are similar to single disk pools, i.e. metadata can survive disk rotting, ditto blocks presence is a requirement for file data self healing.

Multiple disk pools in a redundant configuration (mirror, raidz, raidz2, raidz3) can recover any disk rotting issue (unless of course a massive error situation like multiple disks failing).

Errors are detected when the affected file (or metadata or zvol block) is read. If ZFS can recover the error, the error is fixed transparently and correct data is returned. Otherwise, a read error is reported. Note that the checksum isn't an ECC so it cannot be used to recover broken blocks, only to detect them.

Should you want to verify a whole pool without waiting for a read to occur, you can use the scrub mechanism. ZFS will check all the used blocks and self heal those rotten when possible.

jlliagre
  • 8,861
  • 18
  • 36
  • "A single disk pools can recover a rotten block when it contains metadata." Is this because metadata always has a redundant copy? Whereas data does not without a copies=2 setting? – AaronLS Jun 11 '13 at 00:17
  • Yes, metadata has always one or two copies. https://blogs.oracle.com/bill/entry/ditto_blocks_the_amazing_tape – jlliagre Jun 11 '13 at 05:54
-4

To detect such rotting on an entire disk, you have to run a periodical data srcubbing. Some distribution do it in a cron job, so have a look at it.
The command is zpool scrub techrx. You can run only one scrub process at a time.
Each time datas are read from the disk, zfs check for rotting for the read datas. So you are pretty sure to read clean datas. It is advised to run a complete check once a week up to once a month (as we do for most RAID).

1 you don't need to allocate disk space. The data correction is done within the available space (I remember have seen it's first done by rewriting the erroneous sectors, and in case of failure they are rewriten elsewhere, but this may be inaccurate). Of course, if your disk is already 100% full, this may not be possible
2 the process is based on checksums already integrated into zfs
3 most of the time the scrubbing correct the datas, because the checksum is in fact an error correcting code. if the damaged datas are too big, then zfs can't recover them, but your disk can already considered as dead

You can have more informations here:
run a periodical data scrubbing: http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide
zfs self healing again silent corruptions: http://hub.opensolaris.org/bin/view/Community+Group+zfs/selfheal

Gregory MOUSSAT
  • 1,673
  • 2
  • 25
  • 50
  • I'm afraid both 1 and 3 statements are incorrect. You need to allocate disk space for data errors to be recoverable (even if that disk space can be on the very same disk) and the checksum can only detect bit errors. It is not at all ECC. – jlliagre Apr 22 '12 at 07:37
  • 2
    This is inaccurate. – cstamas Apr 22 '12 at 07:44
  • The first sentence is incorrect too. You don't have to run "zpool scrub xxx" to detect rotting. Rotting detection is always enabled and happen when blocks are read. – jlliagre Apr 22 '12 at 08:03