0

I got a problem after running python script with MongoDB aggregation pipeline. The error said

errno:24 Too many open files, full error: {'ok': 0.0, 'errmsg': 'error opening file "D:\MongoDB\Server\4.4\data/_tmp/extsort-doc-group.463": errno:24 Too many open files', 'code': 16814, 'codeName': 'Location16814'}

The server that host Mongo Database is Windows Server 2016

Problem is gone when I've limited number of data by reducing span day from 7 to 3 days, the script will successfully run and give me a result.

This script have been run for couple weeks before with 7 days setting and there was no problem.

TylerH
  • 20,799
  • 66
  • 75
  • 101
PreuttiP
  • 39
  • 6
  • 1
    Could be difficult, see https://stackoverflow.com/questions/65366248/increase-maximum-open-files-per-session-in-windows-server-2019-mongodb-too-man and https://stackoverflow.com/questions/729162/windows-equivalent-of-ulimit-n – Wernfried Domscheit Jun 24 '21 at 10:09

1 Answers1

0

As per the MongoDB docs here, it is recommended to set a high limit for open file limit. For Ubuntu, we generally do so by changing limits in /etc/security/limits.conf which are specific to user and limit types. There are different ways for different distros. For checking the limits a simple ulimit -a can be very helpful.

IMO machines running databases should have high limits for open file and process count. Also, there is bunch of recommendation from MongoDB related to paging and disk types to use. I would recommend going through them to use MongoDB upto it's potential.

I haven't worked on a windows machine for very long and I am sure if you try to find how to increase open file limit you will find it. Also, when you reduced the query from 7 days to 3 days, the files wired-tiger had to access to fetch the indexes and disk operations also reduced, and it might have allowed you to run the query. Please note unlike some databases, the file system organisation in mongodb-wiredtiger is a bit different.

Sarthak Sawhney
  • 432
  • 4
  • 14
  • ,thank you sir. So in I need to find how to increase open file limit of Windows 2016 then it might help solving this issue. Thank for your advise. – PreuttiP Jun 25 '21 at 05:46
  • is it the way to increase open file limit of Window? _( https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/getmaxstdio?view=msvc-160) _ – PreuttiP Jun 25 '21 at 06:52