2

Reading the Ubuntu Wiki page on ZFS (https://wiki.ubuntu.com/ZFS) I found some claimed abilities of ZFS which I am not able to find any documentation of. If the claims are infact possible I would love to know how to do them

Ari has a single disk workstation. She buys a new disk and plugs it in. ZFS automatically adds the new disk space into the pool. Her home directory is mirrored, while her OS and temp space is striped automatically in the background.

Jack has three disks of different sizes. Configuring a sensible partition and RAID set-up is completely trial and error. ZFS abstracts the three disks into one pool of space, and gives the best balance of performance and security. Jack declares that some media directories do not need to be fault-tolerant, and ZFS transparently stripes them across all the disks.

Both of these quotes seem to indicate that it is possible to have a mix of fault-tolerant regions and high performance regions all in one pool. From every piece of documention I've read this is not possible...does anyone else know a way to do this?

chew socks
  • 212
  • 2
  • 8

2 Answers2

4

What they're alluding to here is a configuration where you simply add all the disks to a pool (without a parent mirror or raidz group controlling the disks).

In this configuration, the number of devices a file is put on is dictated by the copies attribute, which is controlled by the zfs set command - so some data (by ZFS filesystem) in the pool will have higher fault tolerance than other data, and allows fault tolerance across mixed size devices.

Shane Madden
  • 114,520
  • 13
  • 181
  • 251
  • Cool!...no way to do that but with parity instead of mirroring? – chew socks Sep 08 '15 at 22:20
  • @chewsocks Unfortunately, no - the RAID-Z virtual devices are all-or-nothing, there isn't a hybrid "parity-stripe some stuff and don't for other stuff" feature. – Shane Madden Sep 08 '15 at 22:23
  • ...and even though you *can* mix different types of vdevs (stripes, mirrors, raidzN) in the same pool by shoehorning them into the pool, you can't control which data goes onto which vdev. – user Sep 21 '15 at 18:43
3

These paragraphs are misleading and misinformed.

Ari has a single disk workstation. She buys a new disk and plugs it in. ZFS automatically adds the new disk space into the pool.

ZFS doesn't automatically add a newly inserted disk to a pool. You have to run the zpool command to tell how you want the disk to be used.

Her home directory is mirrored

There is mirroring only if Ari asked for but given the remaining of the sentence, that can't be the case.

while her OS and temp space is striped automatically in the background.

If mirroring has been chosen, there can be no striping. If striping has been chosen, there can be no mirroring. In the latter case, there is no "striping in the background" or "post configuration striping" with ZFS. Striping is done once at write time.

Jack has three disks of different sizes. Configuring a sensible partition and RAID set-up is completely trial and error.

Maybe, but easier than with traditional volume management.

ZFS abstracts the three disks into one pool of space, and gives the best balance of performance and security.

Triple mirroring will give both the best read performance and the best security here.

Jack declares that some media directories do not need to be fault-tolerant, and ZFS transparently stripes them across all the disks.

You cannot set properties on directories, you do it with datasets (e.g. file systems) and pools. Creating multiple file systems is a lightweight operation with ZFS though.

To summarize:

You cannot have a mirrored file system and a striped one in a single pool. All datasets belonging to a pool share the same underlying pool RAID configuration.

Assuming ditto blocks (e.g. zfs set copies=2 dataset) are what is used to provide fault tolerance, ZFS will do its best but there is no 100% guarantee all block pairs will end up to a different disks.

Even in the case all ditto blocks are physically located on different disks, the pool would not survive a whole disk failure (or more precisely, would not survive a reboot after a whole disk failure). ZFS copies will protect against limited media errors (see Andreas paragraph) but not a whole disk unavailability. Calling Ari configuration "mirroring" is misleading.

I would never advice such a configuration. Ditto blocks are a great feature if you have a single disk but still want some level of protection against partial media failure (bad blocks) or if you use deduplication on a non redundant pool. Otherwise, expectations shouldn't be set too high and either mirroring or raidz are the way to go with multiple disks pools.

Note that a combination of mirroring, raidz and striping with three different size disks is doable with ZFS by creating multiple partition based pools. For example, let say you have a 4TB, a 3TB and a 2TB disks. You can split all your disks in 1TB partitions, create a 2 TB raidz with 1+1+1 partitions, a 3 TB stripe with 1+1+1 partitions, a 1 TB mirror with 1+1 partition and a 1 TB single device pool with the remaining partition. Do not expect stellar performance with such a mixed raid setup though and strive for the applications not to access concurrently the various pools.

zpool configuration

jlliagre
  • 8,861
  • 18
  • 36
  • Why would it not survive a disk failure assuming the ditto blocks are all duplicated on each disk (in a 2 disk system)? – chew socks Sep 10 '15 at 22:04
  • Because the missing disk will be rightly reported as unavailable. As ZFS doesn't allow to replace an unavailable disk with a spare one, the pool will never came back to the ONLINE state so won't be importable. – jlliagre Sep 11 '15 at 05:22
  • More precisely, the pool should still be usable in a degraded state after the disk failure but won't survive a reboot or an export. – jlliagre Sep 11 '15 at 05:38
  • @jiliagre Ok, this is where I am confused a bit. I ran a RAIDZ2 for several months with one disk removed. When I finally bought a new one it had no problem adding and resilvering despite many reboots. – chew socks Sep 11 '15 at 15:19
  • A raidz configuration is redundant so, like you experienced, supports a whole disk failure (or two whole disks failures in your raidz2 case). The link you refer to is not presenting a raidz configuration but a striped one. It is fooling the readers by talking about mirroring but there is not. As far as the volume layer is concerned, there is no way to import or rebuild such a pool when a disk is missing. I'm not telling using multiple copies is worthless, it definitely helps against partial media corruption, just that the expectations shouldn't be set too high and not the ones of mirroring. – jlliagre Sep 11 '15 at 17:02
  • In the case of Jack, it's worth noting that triple mirroring will allow the vdev to grow only to the size of the smallest constituent disk. (Not that anything else is any better, at least if you want any sort of redundancy. raidz would have the exact same problem.) – user Sep 21 '15 at 18:42
  • @MichaelKjörling yes, whatever the RAID mode chosen (striping, mirroring, raidz*), ZFS best works with equal size disks but only striping avoid wasting disk space if not the case. – jlliagre Sep 21 '15 at 19:28