1

N.B. I'm a complete noob when it comes to ZFS pools.

I created a ZFS pool yesterday across 4 3TB HDDs, like so:

2017-06-10.16:31:08 zpool create data sda sdc sdd sde
2017-06-10.16:31:29 zfs set compression=lz4 data
2017-06-10.16:31:43 zfs set atime=off data

Then I started copying some data in. Awhile later I notice that my pool is gone, and one of my harddrives is offline. I tried rebooting hoping that would bring it back online, but I get stuck in some kind of loop as described here. I didn't have time to deal with it yesterday, so I just shut the machine off until now.

I was about to try and recover from a USB stick, but I tried booting again, and the system came up fine! I run zpool list and it says "no pools available". Did some googling, tried zpool import data. I get an error,

cannot mount '/data': directory is not empty

Tried zpool list again, and sure enough, we're back online:

NAME   SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
data  10.9T   125G  10.8T         -     0%     1%  1.00x  ONLINE  -

Tried ls /data, there's a directory in there, everything looks good. Check my Minio app which stores its data there, and all my buckets are gone. Sure enough, the only thing actually in /data is /data/minio/.minio.sys which the Minio service must have recreated when it came online. zpool history has this scary [scrubbing is OK] looking line in it:

2017-06-11.00:24:08 zpool scrub data

But I don't remember running that. The part I really don't get though, is that zpool list says that 125G is still in use, which sounds about right. So,

  1. Where is that data? How do I get access to it again?
  2. I added ZFS_MOUNT='no' to /etc/default/zfs like that article suggests, but systemctl enable zfs-mount.target says "Failed to execute operation: No such file or directory" -- how am I supposed to automount my ZFS pool such that I won't get stuck during boot?
  3. It appears I created my zpool incorrectly -- via their device names (e.g. sda) instead of UUIDs -- can I correct that now?

My USB stick is assigned a /dev/sdX letter isn't it? I suspect that has something to do with the mounting of the pool.

mpen
  • 423
  • 1
  • 6
  • 15

1 Answers1

3

  • You created a zpool comprised of a stripe of four disks; basically a RAID 0 of disks.

  • You used SCSI /dev/sdX names and added a USB drive into the mix. This can cause SCSI enumeration issues and impact your /dev/sdX drive ordering.

  • If you have something in the /data directory that shouldn't be there, just remove it and attempt to mount your pool - zpool import data

ewwhite
  • 197,159
  • 92
  • 443
  • 809
  • Aha!! It didn't even occur to me that `/data` could have been recreated on a different partition. I'm used to Windows where drive letters usually refer to a physical device. `zpool import data` said "cannot import 'data': a pool with that name already exists". I think `zfs mount` worked though, got my data back, thank you very much! – mpen Jun 12 '17 at 03:19
  • @mpen Drive letters on Windows typically refer to *partitions*, not physical devices. – user Jun 12 '17 at 14:28
  • @MichaelKjörling That's why I said "usually" :-) I know Windows can be partitioned, it's just less common to partition your drives. At least for me I don't bother. One dedicated drive for the OS, and another drive or 5 for data. – mpen Jun 12 '17 at 15:39
  • Will accept this. I never did figure out how to change from "sda" names to device-ids, but I think I needed to redo it to enable mirroring anyway. Seems to be mounting itself correctly now too. – mpen Jun 12 '17 at 15:42
  • @mpen Drive letters on Windows *never* refer to physical devices. Partitioning disks is normally *required* on Windows. You might typically make a single, large partition that covers the entire disk, which may optionally be referred to with a mount point, but in doing so you are absolutely partitioning the disk. If you want to explore the finer points of this further, I recommend asking a new question about it. I won't comment further on this answer on the subject of Windows storage. – user Jun 12 '17 at 16:16