1

Is it possible to configure ZFS to avoid ramdisk usage?

In next scenario:

  • create big file (100GB for example) or a lot of small files (there is much more free RAM than total file(s) size)
  • delete this file

it will be almost no disk IO. Disk IO should happen when there is no free RAM anymore or when cache flushed explicitly (before reboot/shutdown) or data in cache old enough (hours)- it is ok if cache will be flushed from time to time, but not too often (once in an hour is ok). Performance is more important than data integrity.

I could use ramdisk for that, but it will be much more convenient if it possible to configure zfs like that.

ISanych
  • 111
  • 4
  • I'm confused by your question. Are you trying to avoid disk writes, or are you trying to flush to disk immediately? – RobbieCrash Nov 14 '17 at 20:33
  • Avoid. What the point to write a lot of data on slow disk if files will be deleted relatively soon after creation. And because most of data wrtten in docker, lxc & kvm machines I need proper fs backing them. Many reasons: I'm not happy with overlay graphdriver in docker and aufs does not work on debian (proxmox 5) out of the box. Manualy created and managed ramdisk will work, but I'm trying to avoid extra effort and maintaing when containers/vm will need more memory. – ISanych Nov 14 '17 at 20:57

1 Answers1

2

You're better off using a RAMdisk, maintaining it will take a bit of extra effort, but fighting ZFS as much as you'd need to to get it to do this will probably take more.

This is basically totally counter to the way that ZFS wants to work. ZFS will do everything it can to keep your data safe, and then after that it will make your data fast.

If you really want to do this on ZFS it would be really risky and I have no idea if it would actually work. You'd need to create a completely separate Zpool to handle this, and it would probably break completely very quickly after you implemented it. You'd basically need to configure ZFS to hold on to massive amounts of dirty data (sitting only in cache) and you'd be fighting ZFS' desire to keep that data reliable.

You'd need to mess around with setting an insanely high zfs_dirty_data_max limit and set zfs_delay_min_dirty_percent to 100 for the entire pool; throw a tonne of RAM at it; heavily tune the I/O queues for all write operations and the amount of writes you allow it to do at a particular time.

All of that basically would break ZFS' desire to make sure that your data stays safe no matter what happens. In addition you'd probably get your performance totally clobbered whenever you reached those limits and ZFS tried to commit 100GB of buffered data to disk at once, rather than having it make many smaller 100MB writes.

If you really want to try to make it work, take a look at the tunable parameters on ZoL (which I'm assuming you're using rather than Oracle/Solaris ZFS): http://fibrevillage.com/storage/171-zfs-on-linux-performance-tuning

RobbieCrash
  • 1,181
  • 9
  • 26
  • That's the issue - I don't want to delay data writes to disk - I want to avoid most of them (100GB written to fs, but only 100MB written to disk). With ramdrive it will be something like zfs send/receive by cron and before shutdown. I guess easy solution does not exists, but I was hoping that zfs with huge amount of settings have some uknown to me options which will do the magic – ISanych Nov 14 '17 at 23:15
  • I have a feeling that I'll end up with lvm on zram and configure to use proxmox & docker to use these lvms directly – ISanych Nov 14 '17 at 23:57
  • I'm interested in seeing how you solve this, post an update if you remember to once the solution is in-place. @ISanych – RobbieCrash Nov 15 '17 at 23:18
  • Testing thin lvm + zram (had to add types = [ "zram", 252 ] into /etc/lvm/lvm.conf). Looks good so far. In proxmox I could create temporary vms and lxc containers on lvm+zram and permanent vms/lxc on zfs+hdd. So I have added flexibility with that approach. I also should be able to run 2 docker daemons: one for lvm+zram and another for zfs+hdd - but I only read about it and have not tried yet. Unfortunately I could play with that at work only in background and I only shopping for high mem machine for home. – ISanych Nov 16 '17 at 12:37
  • zfs+zram or zfs+brd also should work fine and even easier to maintain for me (as I not very familiar with lvm), but I think it less efficient, as zfs is too sophisticated for ramddrive - maybe I'll test such variant too. – ISanych Nov 16 '17 at 12:38