NodeJS error "EMFILE, too many open files" on Mac OS

Question

For sometime I am having the following error:

Error: EMFILE, too many open files  '/Users/blagus/Gallery/Websites/Nicsware/Pills/resources/core/auth.node.js'
    at Object.fs.openSync (fs.js:427:18)
    at Object.fs.readFileSync (fs.js:284:15)
    at Object.Module._extensions..js (module.js:473:44)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Module.require (module.js:364:17)
    at require (module.js:380:17)
    at instController  (/Users/blagus/Gallery/Websites/Nicsware/Pills/engine/mvc.node.js:79:31)
    at init (/Users/blagus/Gallery/Websites/Nicsware/Pills/engine/mvc.node.js:57:8)
    at route (/Users/blagus/Gallery/Websites/Nicsware/Pills/engine/dispatcher.node.js:268:36)

The line of code making the call to this file (mvc.node.js:79) is

    this.currentRoute.class = require( controllerFile )[dispatchClass].bind( this );

(it is a framework I am creating)

As you can see, the file auth.node.js is called by a REQUIRE, so the given solutions with gracefullFS and similar does not fit. Besides, this problem occour MacOS only. In a Ubuntu seems to work just fine.

Any thoughts?

Is `auth.node.js` doing something strange? If I generate a stacktrace when I require a JS file, `fs.*` doesn't show up. — robertklep, Nov 14 '13 at 16:10
Well, at a Linux box it behaves just fine. Besides is just a simple require that give me trouble — blagus, Nov 19 '13 at 22:06
Looking at the stack trace, the problem isn't caused by the `require` itself but by the module that you're requiring. Then again, if you don't post code it's just a guess... — robertklep, Nov 20 '13 at 06:16

score 57 · Answer 1 · answered Jan 18 '14 at 07:35

57

This worked for me:

ulimit -n 10480

found here

answered Jan 18 '14 at 07:35

awongh

1,308
12
21

14

as Ryan mentioned below. the default limit is 256. setting it to 10480 is probably overkill. Bump it up incrementally `ulimit -n 360` – Lane Jun 25 '14 at 16:13
2

Thanks Lane as well :) anyone know how to kill the processes or close the opened files ? – Jon Apr 11 '15 at 08:26

K M Rakibul Islam · Accepted Answer · 2017-11-06T18:03:29.750

48

You can solve this problem by increasing the maxfiles limit:

launchctl limit maxfiles 16384 16384 && ulimit -n 16384

edited Nov 06 '17 at 18:03

answered Aug 11 '14 at 05:41

K M Rakibul Islam

33,760
12
89
110

4

I won't downvote as this answer is technically correct, but also very dangerous, especially as since Lion the default amount of simultaneous files is along the 10000 mark, setting the limit this low on a running machine can render all open programs unusable in a second. Let's suggest 16384 instead of 2048 instead if we're trying to bump UP the amount – Igor Zinken Aug 13 '14 at 13:20
1

Well, this fixed the issue for me, no more `Error: EMFILE, too many open files` ! Thank you. Also, not sure of what I'm doing, but it works. I would be glad to know the *real* issue behind this problem, why does node pops out this error message. Thank you anyway. – LukyVj Aug 27 '14 at 19:29
I agree w Igor; I am not sure increasing the limit is an acceptable answer for some as this a security risk. I think in reality at some point gulp needs to "let go" of the stream of files. One way to do this is chain some of your gulp tasks via shell scripting . Yes this seems odd but what is the alternative in gulp? She'll or npm tasks/scripts via npm run. – Nick Sep 03 '14 at 17:36
This works well for me but I need to do it after every restart. Is there a way to make it permanent? – OrganicPanda Sep 25 '14 at 09:14
The ulimit value was too large. It threw an error `-bash: ulimit: open files: cannot modify limit: Invalid argument`. So, I reduced it to 8192. Works fine now. – Mike S. Dec 08 '14 at 16:41
if you add it into ~/.bashrc 'launchctl limit maxfiles 16384 16384 && ulimit -n 16384' it will be permanent, (bashrc is ran everytime you start a terminal (bash) session – aqm Jan 16 '15 at 09:33
2

You have to put sudo at the beginning. It didn't work until I added sudo – Hamid May 19 '15 at 14:54
2

This did not seem to do anything for me. – ryanwebjackson Aug 08 '18 at 19:37

score 20 · Answer 3 · edited Oct 05 '20 at 11:40

20

i had this error and the ulimit and launchclt didn't work for me,

this solution from http://yabfog.com/blog/2014/10/22/yosemite-upgrade-changes-open-file-limit worked for me

echo kern.maxfiles=65536 | sudo tee -a /etc/sysctl.conf
echo kern.maxfilesperproc=65536 | sudo tee -a /etc/sysctl.conf
sudo sysctl -w kern.maxfiles=65536
sudo sysctl -w kern.maxfilesperproc=65536
ulimit -n 65536 65536

and then putting the

ulimit -n 65536 65536

into ~/.bashrc

edited Oct 05 '20 at 11:40

NearHuscarl

66,950
18
261
230

answered Jan 16 '15 at 10:45

aqm

2,942
23
30

should mention it's a hacky solution but one to helps out when your local machine isn't working well – aqm Nov 08 '20 at 22:19

score 17 · Answer 4 · edited Aug 04 '23 at 17:01

17

I am using watchman. Which fixed this error for me. Worth trying!!!

brew update
brew install watchman

It watches the files and documents and triggers file handling events if there are any changes. Here's their github repo:

https://github.com/facebook/watchman

edited Aug 04 '23 at 17:01

JimmyT

759
5
7

answered Aug 15 '18 at 06:40

bh4r4th

3,760
1
21
25

3

This is the only one that worked for me, on Catalina 10.15.1. – Daniele Molinari Nov 13 '19 at 11:48
1

Catalina 10.15.3 here as well - only solution that worked – Imre Kelényi Apr 15 '20 at 10:06
4

Can you provide some context as to how this works? Posting two installation commands is not helpful. – Jun 16 '20 at 15:30
1

Hi @Contrapak As the name says, it just watches the files and documents. Triggers file handling events if there are any changes. Please look into their github repo for better understanding. https://github.com/facebook/watchman – bh4r4th Jun 22 '20 at 04:27

score 9 · Answer 5 · answered Nov 22 '13 at 21:44

Your code is opening too many files. By default, OS X has a limit of 256 simultaneously opened files. When your code requires a new module, node has to open the file to read it in. If you are already at this limit, node's require cannot continue and will throw an Error. You should audit places in your application where you are calling fs.open and ensuring that you are properly closing all of those files. You may also encounter this problem if you attempt to do too many simultaneous file system reads, since each pending read will be an open file. I have also encountered this problem while using fs.watchFile, which also requires opening a handle to the file.

Do you have any resources to make sure I'm doing what you mentioned? I would prefer to practice better usage of fs instead of just having to keep upping the limit as others have suggested. — Katrina, Apr 12 '16 at 18:21

score 7 · Answer 6 · answered Oct 06 '15 at 08:25

7

Check your ulimit. For example initialy my ulimit on OSX was 256.

Run ulimit -n to see the limit.
Afterwards You can ulimit -n 1024 to set a higher limit.

answered Oct 06 '15 at 08:25

Koray Güclü

2,857
1
34
30

score 6 · Answer 7 · edited Jun 08 '15 at 12:09

6

None of the other answers worked for me. This did the trick:

launchctl limit maxfiles 16384 16384

Also to note, this doesn't save across sessions so unless you want to run it for each bash terminal session I suggest putting the above line in your ~/.bashrc (or ~/.zshrc if you are using zsh) by doing this at the command line:

vi ~/.bashrc

edited Jun 08 '15 at 12:09

RedYeti

1,024
14
28

answered Oct 22 '14 at 16:59

John Culviner

22,235
6
55
51

score 2 · Answer 8 · edited Jun 08 '15 at 12:29

ulimit is great if you are using the terminal but it only works if you are running your app from the same terminal tab ( or shell instance ). Launchctl is great but is systemwide. If you leave Launchctl limit maxfile alone, the soft limit is 256 and the hard limit is unlimited.

In a production environment, you will probably need to launch at startup and reboot on crash, which means the best answer for Mac OSX is to use a .plist file for each of your applications. I launch my node application using said plist file ( which runs at start up and reboots after crashing )... inside this file you can set the amount of files per application using the SoftResourcesLimit key.

<key>KeepAlive</key>
<true/>

<key>RunAtLoad</key>
<true/>

<key>SoftResourceLimits</key>
<dict>
<key>NumberOfFiles</key>
  <integer>16384</integer>
</dict>

score 1 · Answer 9 · answered Sep 24 '15 at 17:19

Maximum files was reseted to 256 in OS X 10.10.3 Yosemite. This can lead to problems with npm installations. You can check this limit from terminal with command ulimit -n. In order to change this beyond 256, you need to create two configuration files.

The first property list file /Library/LaunchDaemons/limit.maxfiles.plist :

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
 <plist version="1.0">
    <dict>
      <key>Label</key>
        <string>limit.maxfiles</string>
      <key>ProgramArguments</key>
        <array>
          <string>launchctl</string>
          <string>limit</string>
          <string>maxfiles</string>
          <string>65536</string>
          <string>65536</string>
        </array>
      <key>RunAtLoad</key>
        <true/>
      <key>ServiceIPC</key>
        <false/>
    </dict>
  </plist>

The second property list file /Library/LaunchDaemons/limit.maxproc.plist :

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple/DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
  <plist version="1.0">
    <dict>
      <key>Label</key>
        <string>limit.maxproc</string>
      <key>ProgramArguments</key>
        <array>
          <string>launchctl</string>
          <string>limit</string>
          <string>maxproc</string>
          <string>2048</string>
          <string>2048</string>
        </array>
      <key>RunAtLoad</key>
        <true />
      <key>ServiceIPC</key>
        <false />
    </dict>
  </plist>

Set the proper ownership and rights:

sudo chown root:wheel /Library/LaunchDaemons/limit.maxfiles.plist
sudo chown root:wheel /Library/LaunchDaemons/limit.maxproc.plist
sudo chmod 644 /Library/LaunchDaemons/limit.maxfiles.plist
sudo chmod 644 /Library/LaunchDaemons/limit.maxproc.plist

Set the desired limits to bash profile file (.bashrc or .bashprofile or similar):

ulimit -n 65536
ulimit -u 2048

Make sure that the rights are the same for bash profile:

chmod 644 .your_bash_profile_file

Restart computer and check with ulimit -n max files. It should be 65536 and you should be able to change it anything below that.

Source: http://docs.basho.com/riak/latest/ops/tuning/open-files-limit/#Mac-OS-X

score 0 · Answer 10 · answered Jun 02 '16 at 19:12

0

awongh's answer worked for me

ulimit -n 10480

But only after starting an interactive shell

sudo -i

Outside of the shell I kept getting a permission error on OSX Yosemite

answered Jun 02 '16 at 19:12

withchuck

1
1

score 0 · Answer 11 · answered Jul 31 '18 at 09:27

0

As for other questions about EMFILE error, you have to manage a queue to limit number of files opened at the same time. Increasing limit will only delay your problem.

There is some responses there : node and Error: EMFILE, too many open files

answered Jul 31 '18 at 09:27

Plaute

4,669
2
17
19

score 0 · Answer 12 · edited Apr 15 '22 at 22:43

0

You need to reinstall watchman and make sure that brew's version is the one that's gonna be used in the terminal. Just follow these steps in order to do it:

brew update
brew install watchman
brew link --overwrite watchman

edited Apr 15 '22 at 22:43

Skatox

4,237
12
42
47

answered Apr 12 '22 at 16:21

Anandu K Dev

21
2

Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Apr 12 '22 at 19:02

NodeJS error "EMFILE, too many open files" on Mac OS

12 Answers12

Linked