2

I have a production machine at work that was running on a single 3TB HDD. Now I've added another 4TB drive into it. The 3TB drive has been used for 1,5 years at this point, the 4TB is brand new.

I have about 1500 GB of unique data (ignoring backups) on the 43 TB drive at this point. I'll be storing about 500 GB of archives from other machines on the network, giving the Samba shares for accessing their old files. It doesn't need to be fast, but should be redundant (I have the only copy of the data).

I'm not sure how I should organize the storage to maximize security of the data with what I have right now. I will probably be able to add another 4 TB drive to this machine in 2 years, it'd be good to be ready for that.

I basically need 3 different types of storage:

  1. System - place to boot and run my production Linux OS
  2. Data - place to store current projects and files served to other users on the network
  3. Archive - place to keep old projects (preferably with compression)
  4. Backup for system and current data, should I blow anything up by accident at any time on my main pool.
  5. Hopefully some redundancy for archive as well (as these will be only copies of old projects)

I'm thinking about partitioning my 4TB disk: 1 TB for SWAP, GRUB and Linux OS 3 TB for ZFS And adding 3TB mirror across the whole 3TB drive for redundancy.

ZFS snapshots could to some degree replace my usual production files backups, that'd save me a lot of space probably. However - if anything happens to the Zpool, I am in big trouble.

I know I can create single disk vdev (on a 3TB partition on the 4TB drive) I can then move there all my data, freeing up my 3TB drive. I could prepare a boot and swap partition (not sure how to do this though right now though) in the last 1TB of my 4TB disk, the clear my 3TB drive, and add it to my zpool as mirror, gaining redundancy.

Does this sound like a good idea?

I have about 12 years experience using Linux, I have built my own Kernel and installed Gentoo, I use it daily for all my work with graphic design and media production, I do a bit of system administration at the company where I work, but I would by no means call myself as sysadmin. I don't have a lot of ZFS experience - I mostly deployed it for fun on USB flash drives and USB hard disks in the past few months, trying to learn about it as much as I can.

An alternative is to just use the 4TB drive as an archive + backup storage with EXT4 + SWAP (to reduce the stress on the main drive - the SWAP is particularly been grinding hard at times for the past 1,5 years).

That could be a better idea, as it's way simpler and doesn't require me to move all my system to a new drive, and I still have redundant copies (though no filesystem compression or checksumming). Also I know the 3TB drive will probably die first so I'd rather move everything to the new 4TB one to put less stress o n the old 3TB one, so it can last longer.

unfa
  • 171
  • 1
  • 8

1 Answers1

2

There are a couple of ways you could go about using the second 4TB drive. One possible way is what you mentioned, setup the new 4TB drive and then copy the data. However I suggest this other approach.

  • Copy all your data from the 3TB to the 4TB
  • Setup the original 3TB as boot with ZFS
  • Copy the data from the 4TB back to the 3TB (now with ZFS)
  • Setup the 4TB as mirror of the 3TB. You would lose 1TB of usage from the 4TB, but later when you get another 4TB you can grow the vdev

PROs:

  • You have your data in a safe state at every step of the operation. To be safe suggest you use rsync with checksum flag every time you are copying the data around.
  • You end up with RAID 1 at the end

CONs:

  • You lose 1TB from the 4TB. You may be able to install ZFS in a 3TB partition, but that would limit your ability to grow ZFS later
  • It will take several rounds of copying your data around so may be a bit of time (mostly machine time though)

If the data is really important I would also suggest to use a backup to either a 3rd drive, if you have one, or to a cloud service like B2. You mentioned 1500GB. Looking at B2 pricing looks like it would be about $7.5 for that amount of data. Once you have finished the operation and you are on your end state (RAID 1 on ZFS) you could delete the backup on B2 if you wanted (although would likely be a good idea to just keep backing up). https://www.backblaze.com/b2/cloud-storage-pricing.html

Through this exercise the most important thing to try to make sure is that your data is safe. Right now you only have it on one drive so long term doing nothing is not good either.

Going with ZFS gains you snapshots and RAID1 which can be very useful in comparison to say putting the second drive in and just rsyncing to it or using it as extra storage. Snapshots are a great space saver when it comes to doing backups; specially if a lot of your data doesn't change.

Francisco1844
  • 260
  • 2
  • 8
  • I have created a 3TB partition on the 4TB drive matched in size with my 3TB drive, and copied my data to ZFS datasets into that pool using copies=2 to be sure I don't loose anything to bitrot. I have already tested the 4TB drive with `badblocks -vwp 4` and S.M.A.R.T - all good. I wonder now - I if I move my OS to the 4TB drive, and then add the 3TB drive as a mirror. I also should be able to remove the 4TB drive partition from the pool, and then re-add it as a whole disk, right? If I then do the same with my 3TB drive (remove it as a mirror) and swap it out for another 4TB drive... – unfa Feb 09 '18 at 12:02
  • 1
    What you did is different from what I initially suggested and in my opinion potentially riskier. This part in particular: remove the 4TB drive partition from the pool, and then re-add it as a whole disk. In my opinion the least you move things around the safer your data is. You have more chances for issues the more you move the data around back and forth. What I had suggested was to NOT put ZFS in the 4TB, copy the data to it, then redo the 3TB with ZFS on root, copy the data and THEN put the 4TB. Main distinction is that you want to do the OS and then attach the remaining drive. – Francisco1844 Feb 09 '18 at 14:50
  • I've done this mainly as a temporary test to see how it'll behave, I will probably work to implement what you have proposed. I yet have to make sure I can setup a working ZFS bootable root filesysystem. – unfa Feb 13 '18 at 13:23
  • Ok, so I've moved my files to a ZFS pool on the 4TB drive. Then I reformatted the 3TB drive and created a Zpool partition there. I attached it with `zpool attach` command to have in in mirror. After resilvering, I've chrooted into my zpool, installed `zfs-initramfs` package, installed and updated GRUB from there and it works! – unfa Feb 22 '18 at 11:16