0

I've read that "It is not recommended that RAID types be mixed in a pool", which implies that it is possible. Why is it not recommended?

It would be very desirable in my setup, which consists of a mix of very-important, important and not important files, to dynamically assign different RAID-Z levels for different file systems inside ONE pool.

The second question is: what would happen, if disks begin failing? I understand, that I wouldn't be able to access not sufficiently raided file systems. But what if the raided ones (with sufficient redundancy to handle the failure) are nested inside file system that failed? Would they by available? How?

I'm planning to use ZOL, version 0.6.0.80, which compatible with zpool version 28 and zfs version 5.

Adam Ryczkowski
  • 720
  • 1
  • 9
  • 29

2 Answers2

5

Your point about storing important files on the more secure file systems shows an incorrect idea of how ZFS stores data within the pool.

If you have multiple vdevs, your data for all file systems is striped across all of the vdevs, you can't assign a file system to a specific vdev, the pool is used as a whole.

This means that the entire pool is only ever as safe as the weakest vdev. If you had a pool made up of 5 radiz3 vdevs, and a single disk vdev, a failure of that single disk would bring the entire pool down.

If you want multiple levels of safety, you're better off creating separate pools.

USD Matt
  • 5,381
  • 15
  • 23
  • So... does it mean, that because the vdevs cannot be resized, I must declare beforehand the size of raid-z2, raid-z1. I cannot postpone this decision to the time when the file systems are built? – Adam Ryczkowski Sep 28 '12 at 15:07
  • Yes you cannot resize the vdevs in the pool once it is made, only add more vdevs (which should ideally match the original). The ability to create multiple file systems doesn't really create separate file systems in the traditional sense. Separate file systems on ZFS are really more of a management thing. You can specify different mount points, configure specific ZFS properties and take snapshots at a file system level, but the actual data is all stored together on the pool. – USD Matt Sep 28 '12 at 15:39
  • @AdamRyczkowski The most flexible solution is to use mirrors. The RAIDZ solutions aren't resizable. What are you looking to do? – ewwhite Sep 28 '12 at 18:33
  • @ewwhite Well... I was thinking about ability to change raid levels per-file or per-directory *after* the file system is created. Imagine, that I downloaded one file, which wasn't that important, but with time the source went off line, and I started to depend on existence of this file. So I would change the raid level to make sure this file would survive hard disk failure. – Adam Ryczkowski Sep 28 '12 at 18:56
  • 1
    @AdamRyczkowski You can modify the [`copies` parameter](http://serverfault.com/questions/377892/zfs-how-do-you-restore-the-correct-number-of-copies-after-losing-a-drive) for a particular filesystem. That means you can make sure multiple copies of the files contained within exist. See my answer below. – ewwhite Sep 28 '12 at 18:58
  • Please note that copies is most useful for single disk systems to allow files to automatically be recovered in the event of bad blocks, checksum errors etc. If you lose one disk or have errors in a raidz pool, the redundancy will keep the data available, if you lose 2 the pool will be faulted and all data inaccessible regardless of the copies setting. From what I can see this setting is generally a waste of time thats likely to get people into trouble and you should just rely on redundant pool configurations if possible. – USD Matt Sep 28 '12 at 19:17
2

Just because you can doesn't mean you should...

How would you assign this data to the desired RAID sets dynamically?

The reason for avoiding RAID types in a pool is consistent access time/speed and reliability.

Your top-level devices, vdevs, end up being striped. ZFS performance scales with the number of stripes. While you could have a vdev of RAIDZ1 and a vdev of RAIDZ2, if combined in a pool, they would be striped together (e.g. RAID0 across them). You have no control over where or how the data is written...

If you have enough disks to have different vdevs and RAID types, please separate them into multiple pools.

There is a feature to allow you to set the number of copies of files so that they can be stored in different parts of the filesystem. Maybe that's closer to what you need.

Take a look at this resource to help you plan your ZFS implementation.

Paraphrased suggestions...

  • Do not mix disk sizes or speeds within a single vdev
  • Do not mix disk sizes or speeds within a zpool at all
  • Do not mix redundancy types within a zpool - if you do mirror vdevs then do all mirror vdevs, if you do raidz2 then do all raidz2
  • Do not mix disk counts across vdevs within a zpool
Community
  • 1
ewwhite
  • 197,159
  • 92
  • 443
  • 809
  • As I understand the referenced post "number of copies of files", it will not help me to recover that files in case of total disk failure, no mater how many copies there were, will it? – Adam Ryczkowski Sep 28 '12 at 19:36
  • ZFS may spread copies(ditto blocks) on different vdevs, but it's unclear if it'll spread copies to vdevs first before writing to the same vdevs. http://superuser.com/questions/194644/freenas-single-disk-zfs As for recovery, I'm still trying to figure out if it's possible to recovery from a spanning non-redundant zpool with some subvolumes with copies=2 turned on. It seems to be possible, but I still need to do some of my own testing to make sure. http://serverfault.com/questions/377892/zfs-how-do-you-restore-the-correct-number-of-copies-after-losing-a-drive – goofrider Oct 23 '12 at 20:20
  • Oh never mind, found it. ZFS will store ditto blocks on separate vdevs. https://blogs.oracle.com/bill/entry/ditto_blocks_the_amazing_tape – goofrider Oct 23 '12 at 20:42