0

I am using Bazaar v2.0.1 on Max OS X 10.6.2

When I perform a commit after moving a large number of files/directories (over 10,000) I get the following error message:

bzr: ERROR: [Errno 24] open: Too many open files: '.'

My first work-around was to break the commit up into several sub-sets. However, this is not ideal and I'm afraid there may be a point where one change (that cannot be broken up into sub-sets) will give me the same error.

[Update]

After doing some research this is what I have found:

It looks like:

Errno 24 "open: Too many open files"

is a Python error.

According to this blog post, the limit on the number of files open can be changed from within a Python script with resource.setrlimit. However, I was really looking for a way to change the default value so Bazaar would automatically run with a higher value (BTW, it looks like my default setting was 2560).

According to the apple documentation for the setrlimit system call there is a sh built-in command called ulimit which can be used to change the setting. Any process started from the shell would then inherit this value.

My current work-around is to add ulimit -n 10240 to ~/.profile. This way when I run bzr commit from the shell it will be able to open 10240 files. I selected 10240 files because this is the maximum allowed for a user process in Mac OS X.

It doesn't seem like Bazaar should need that many files open at once. I am worried that if I ever move more files that this may come back to bite me again. Is this a bug in Bazaar? Is there anything else I can do?

Trent
  • 13,249
  • 4
  • 39
  • 36
  • 1
    When you say large number of files/directories, approx. how many? Hundreds? thousands? – Frank V Jan 18 '10 at 21:32
  • 1
    Close some files? (just kidding ;) – Cyclone Jan 18 '10 at 21:34
  • @Frank, I believe there are over 10,000 files that have been moved. But why would Bazaar need those all open while performing the commit? the files have already been moved in the file system. – Trent Jan 18 '10 at 21:47
  • I suspect Bazaar would open the files to read the contents to store the differences. Why would it keep them open is, I think, the question... Bug? – Frank V Jan 19 '10 at 03:21
  • It's also not uncommon for folks who are writing Python code to rely on the garbage collector to close files they're done with; in this case, the buggy application doesn't necessarily keep _all_ files it ever touched open, just however many are associated with objects not yet caught by a full GC run. If this is the case, bumping up the limit may be a fine workaround (though you really should file a ticket with the Bazaar folks on this). – Charles Duffy Jan 19 '10 at 22:10

2 Answers2

3

You can use lsof to see all open files. You might try grepping for the pid of the bazaar process, or monitoring the number of open files.

Note that you may or may not need to be root to see all files / processes relevant for your situation.

Kaleb Pederson
  • 45,767
  • 19
  • 102
  • 147
  • Ok, so if I get the list of files that bazaar has open how does that help? It is still going to error out. – Trent Jan 18 '10 at 21:48
  • It helps when there's a bug or other apps are misbehaving and stealing all your file descriptors. I actually never considered that your limit was set low enough to cause a problem. – Kaleb Pederson Jan 18 '10 at 21:58
2

Try ulimit -n 1024 (or more) before running bazaar, if your shell supports it (it's a bash builtin).

Jinx! edit: you can put it in your ~/.profile if there is one, or ~/.bash_profile.

Tobu
  • 24,771
  • 4
  • 91
  • 98