0

I'm experimenting with Native-ZFS on Ubuntu right now. Here are the drives installed on the system:

  • 2 x 2TB
  • 3 x 1TB
  • a 200GB operating system disk

I've got the OS installed and the stable ZFS RC for 12.04 installed via the PPA.

In terms of ZFS configuration, I'd like to get the maximum theoretical capacity with 1 drive failure protection (so 5TB). I was planning on this configuration:

  • 1 zpool:
    • 1 4TB RAIDZ vdev:
      • 3 x 1TB drives
      • 2 x 1TB partitions, one from each of the 2TB drives
    • 1 1TB Mirrored vdev:
      • 2 x 1TB partitions, one from each of the 2TB drives

First off, does this configuration make sense? Is there a better way to achieve 5TB (such as a 7 x 1TB RAIDZ2)? I'm not terribly concerned with performance (although I am somewhat concerned with upgradeability).

Secondly, can anybody point me to a guide (or show me) the ZFS incantations to create such a (mildly complicated) pool? All of the guides I've found create a 1-1 zpool-vdev and use the entire raw disk, not partitions. Most of the documentation I've found for ZFS regarding partitions is BSD or Solaris dependent, and I'm not sure how much of it applies to Linux.

Thanks!

Ryan N
  • 221
  • 1
  • 7
  • 1
    Is there a reason you're using ZFS in Ubuntu? There's no true "native" linux port, afaik ZFS support on Ubuntu is a userland implementation, not a native kernel one. I'd recommend FreeBSD or OpenIndiana if you want a truly native implementation of it. Unfortunately, the Linux kernel being encumbered by the GPL prevents a true native ZFS implementation under ZFS's current license. – MDMarra Jul 19 '12 at 01:16
  • 2
    I used to use OpenSolaris for precisely that reason, but this project is quite mature now: http://zfsonlinux.org . Of course it can't SHIP with the kernel, but that doesn't mean it can be installed as an add-on kernel module. – Ryan N Jul 19 '12 at 01:19
  • If it's a "native" port, shouldn't the commands all be the same as OS/OI/BSD? – Chris S Jul 19 '12 at 01:21
  • Hm, I hadn't realized that there was a stable, production-ready kernel module for ZFS on Linux. – MDMarra Jul 19 '12 at 01:21
  • The commands are the same, but I'm confused on how to make the partitions, as all of these OSs have different partitioning tools. What flags should I set, should I set an initial filesystem on them, etc. – Ryan N Jul 19 '12 at 01:25
  • _ Is there a better way to achieve 5TB (such as a 7 x 1TB RAIDZ2)_ -- Yes, using 7 x 1TB is better then using 2TB drives for both a RAIDZ vdev and Mirrored vdev. If one if your 2TB drives fail, you'll have two degraded vdevs. Not to mention this will confuse your coworkers. – Stefan Lasiewski Jul 19 '12 at 01:29
  • @RyanN That disk geometry is less than ideal.. 2 disks getting double the I/O of the other 3 will bottleneck you in nasty ways. Why not just keep the groups to like-sized drives (3-disk RAIDZ1, 2-disk mirror)? You lose 1TB, but it'd be simpler and faster. – Shane Madden Jul 19 '12 at 01:31
  • @StefanLasiewski I remember reading on Wikipedia that using multiple vdevs in a pool allows for more than one synchronous write. Also, wouldn't there be faster random reads from the partitions in the mirrored zpool than from those partitions if they were in a RAIDZ? – Ryan N Jul 19 '12 at 01:42
  • @ShaneMadden that extra TB would likely mean a good deal more time until I have to upgrade, it's kind of worth it to me. This is mostly a seldom-write, often read scenario. – Ryan N Jul 19 '12 at 01:47
  • Just as an aside, I have Ubuntu Linux boxes set to EFI boot from USB sticks but with the root filesystem in the ZFS pool, benefiting from RAID, snapshots, etc. so you don't necessarily need a separate system disk. – rptb1 Oct 05 '18 at 15:25

1 Answers1

1

the only difference between using the whole disk to create a pool and part of a disk is that you have to partition the disk first. so on your 2 TB drives, create 2 partitions, each 1 TB, using whatever partition tool you choose. (that would not be a zfs utility, but instead something like fdisk.)

then when you issue your zpool command, pass the partition instead of the drive:

zpool create tank1 raidz /dev/sda /dev/sdb /dev/sdc /dev/sde1 /dev/sdf1

and the same for the 1 TB mirror:

zpool create tank2 mirror /dev/sde2 /dev/sdf2
longneck
  • 23,082
  • 4
  • 52
  • 86
  • That makes sense. Two things: 1. If the second command were `zpool add tank1 mirror /dev/sde2 /dev/sdf2` would we have one zpool with two different vdevs in it? 2. With Linux, I read that one should use /dev/disk/by-id handles instead of /dev/sd* handles. Partitions on the disks should show up as handles in /dev/disk/by-id/, right? – Ryan N Jul 19 '12 at 01:46
  • @RyanN Right on both counts! An scanning import seems to always find the right devices (though I've always used full disks, not partitions), so I'm not sure how necessary the use of `/dev/disk/by-id/` really is, but yup, `-part1` etc are appended to partitioned devices there. – Shane Madden Jul 19 '12 at 01:54
  • I just tried that and I got the following error: `mismatched replication level: pool uses raidz and new vdev is mirror`. Shouldn't the pool allow multiple vdevs of different types? – Ryan N Jul 19 '12 at 02:21
  • 1
    Ah, see [here](http://www.ronmar.netfirms.com/ppc/linux2/unix-linux/chapter8.html) "zpool initially balks at this configuration because the two virtual devices have different redundancy schemes. This particular configuration is OK since both vdevs have some redundancy. In actual use, you should not mix redundant and nonredundant vdevs since there’s no way to predict which blocks might be stored on which devices; partial redundancy is useless. " – Ryan N Jul 19 '12 at 02:22