1

I have a pool of 5 x enterprise 12Gps SSD that I use for internal storage for a proxmox server. This pool is used to stand up VMs and containers. The problem is that it is eating up a lot of my available RAM which is 192G! Here is the summary: enter image description here

As you can see the Max size is 94.4GiB!!!

My question is: What should be the optimal value for this? and how to change it?

This is my production server and I want to be 100% sure of the process. (I already looked here but my server is also not honouring the arc size :( )

Any help or suggestions would be highly appreciated!

Shery
  • 121
  • 1
  • 6
  • *I already looked here but my server is also not honouring the arc size :(* Try `echo 3 > /proc/sys/vm/drop_caches` That should cause the system to drop the ZFS ARC along with the page cache. Once the ARC is below its new limit, it should honor that limit. – Andrew Henle Dec 02 '21 at 22:44
  • And a large ARC doesn't have to be a problem - it's only a problem if your system has sudden demands for large chunks of memory, especially huge pages. Because ZFS ARC release can be accurately timed with a sundial. So when you get a sudden demand for a lot of memory, if that demand can't be met without releasing a big chunk of the ZFS ARC, your entire system can stall while the VM system thrashes about waiting for ZFS. – Andrew Henle Dec 02 '21 at 22:48

2 Answers2

1

You can persistently reduce the ARC size by doing the following.

Edit or add a file named

/etc/modprobe.d/zfs.conf

Add a line as follows. This example line is for 1 gig of memory. So multiply the number by the number of gigs you want it to be sized.

options zfs zfs_arc_max=1073741824

After saving the file run the command

update-initramfs -u -k all

Then reboot to apply the changes.

Chris C
  • 71
  • 2
0

By default, ARC uses up to 50% of system RAM. However, ARC is dynamic: it will tune itself down if the system experience increased memory pressure.

So I suggest capping it down only if you experience some real system issue, or in system with really large RAM where having so much cache is of no real advantage.

That said, you can manually (and dynamically) cap ARC via the zfs_arc_max tunable. However, be sure to understand that simply reducing it below current ARC usage will not immediately release memory. From the man page:

zfs_arc_max can be changed dynamically with some caveats. It cannot be set back to 0 while running and reducing it below the current ARC size will not cause the ARC to shrink without memory pressure to induce shrinking.

If you need to immediately free ARC memory, you can set the tunable above and then manually dropping cache via echo 3 > /proc/sys/vm/drop_caches.

Alternatively, you can permanently set it via /etc/modprobe.d/zfs.conf and reboot your system.

shodanshok
  • 47,711
  • 7
  • 111
  • 180