Setting maxfile on OS X/MacOS Programmatically

Question

If I look at the maxfiles option using launchctl's limit command (on my OS X El Cap machine)

$ launchctl limit
    cpu         unlimited      unlimited      
    filesize    unlimited      unlimited      
    data        unlimited      unlimited      
    stack       8388608        67104768       
    core        0              unlimited      
    rss         unlimited      unlimited      
    memlock     unlimited      unlimited      
    maxproc     709            1064           
    maxfiles    256            unlimited

There appears to be a soft limit of 256, and a hard limit of "unlimited". I'd like to change the soft limit to be something like 2048, and leave the hard limit untouched. When I look at limit's arguments

$ launchctl help limit
Usage: launchctl limit [<limit-name> [<both-limits> | <soft-limit> <hard-limit>]

It appears I can either set both limits to the same thing, or set a value for soft and hard. However, if I attempt to set a hard limit of unlimited.

$ sudo launchctl limit maxfiles 2048 unlimited

I end up with the curious value of 10240

$ launchctl limit
    cpu         unlimited      unlimited      
    filesize    unlimited      unlimited      
    data        unlimited      unlimited      
    stack       8388608        67104768       
    core        0              unlimited      
    rss         unlimited      unlimited      
    memlock     unlimited      unlimited      
    maxproc     709            1064           
    maxfiles    2048           10240

What's going on here? Is it possible to set a value of unlimited? If not, is that a limitation of the launchctl limit command, or something on the system level? If the later, what are all those initial value of unlimited reporting? Or is this Apple being Apple?

For bonus points -- does anyone know why this limit is set so low in the first place?

Why does the subject use the word "programmatically"? What you're showing here is not setting the limit programmatically in the conventional sense (i.e. by calling a system API within your own program). You're invoking "external" tools. As such, it's bordering on off-topic and should maybe be on Super User or Ask Different. — Ken Thomases, Mar 11 '17 at 22:12
@KenThomases The is the question boiled down to its essence, my actual context is using these commands as part of a shell script I'm programming, and I'm interested in the underlying system calls as well. i.e. I'm not committed to using launchctl in my program, it's just the most convenient mechanism right now. No argument that it's borderline, but all my years of SE involvement make me feel like it's on the side of the border that says let it stay :) — Alana Storm, Mar 11 '17 at 22:42
I tried to change this limit using that 'sudo'd' command, but I found it effectively disabled not just my terminal session but everything on my mac with a 'too many files open' error message, and I had to do a hard reboot (i.e. power switch) to get things back the way they were. and it did not change from 256 either. A word to the wise ;-) — George D Girton, Jul 29 '21 at 18:39

score 3 · Accepted Answer · answered Mar 11 '17 at 22:07

From the man page for setrlimit(), the underlying system call that launchd would rely on to implement this feature:

setrlimit() now returns with errno set to EINVAL in places that historically succeeded. It no longer accepts "rlim_cur = RLIM_INFINITY" for RLIM_NOFILE. Use "rlim_cur = min(OPEN_MAX, rlim_max)".

So, while getrlimit() may (initially) report that the hard limit is RLIM_INFINITY, it is, practically, OPEN_MAX (10240) because that's simply the limit imposed by how the kernel is implemented. If you attempt to set the limit, you'll need to use the latter value, which then affects what's reported from then on.

As to why the soft limit for RLIM_NOFILE defaults to 256, it's because for the vast, vast majority of processes, this is not a practical constraint. They get along just fine without coming close to this limit. And keeping the value "so low" means that each process has a lower cost in terms of memory and each fork() has less work to do to duplicate file descriptors into new child processes, etc.

Programs which know they are likely to work with many more files can change their own limits using setrlimit().

score 2 · Answer 2 · answered Mar 11 '17 at 20:35

Without speculating as to why, Apple changed these in Yosemite. Any changes made via launchctl limit ... are temporary to your current login only. Relogin or reboot and they go right back to the defaults Apple sets.

I can confirm they have not changed this behavior yet as of 10.12.4 b4.

An unfulfilling digestible when it's Apple being Apple. I did a deep dive in the developer docs and could not find an explanation for why they made the change.

Setting maxfile on OS X/MacOS Programmatically

2 Answers2