2

I'm curious what limit I'm raised? I have the next code:

import httplib, resource, socket, traceback

print("NOFILE: %s" % str(resource.getrlimit(resource.RLIMIT_NOFILE)))
socket.setdefaulttimeout(100000)

conns = []
for _ in xrange(10000000):
    con = httplib.HTTPConnection('ya.ru')  # or 93.158.134.3
    try:
        con.connect()
        conns.append(con)
    except:
        print("Total connections: %s\n" % len(conns))
        print(traceback.format_exc())
        break    
input("Press any key to exit...")

When HTTPConnection uses a host name, I get the next output:

NOFILE: (2560, 9223372036854775807)
Total connections: 1019

Traceback (most recent call last):
  File "test.py", line 13, in <module>
    con.connect()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 772, in connect
    self.timeout, self.source_address)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 571, in create_connection
    raise err
error: [Errno 65] No route to host

And when I use the direct IP address:

NOFILE: (2560, 9223372036854775807)
Total connections: 1021

Traceback (most recent call last):
  File "test.py", line 13, in <module>
    con.connect()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 772, in connect
    self.timeout, self.source_address)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 571, in create_connection
    raise err
error: [Errno 36] Operation now in progress

It looks like pretty the same point of failure because 1019, 1021 and 1024 are very close values. I think 1024 is a some limit within my OS. 

uname -a
Darwin hackmachine 13.3.0 Darwin Kernel Version 13.3.0: Tue Jun  3 21:27:35 PDT 2014; root:xnu-2422.110.17~1/RELEASE_X86_64 x86_64

ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
file size               (blocks, -f) unlimited
max locked memory       (kbytes, -l) unlimited
max memory size         (kbytes, -m) unlimited
open files                      (-n) 2560
pipe size            (512 bytes, -p) 1
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 709
virtual memory          (kbytes, -v) unlimited

So, my questions are:

  • What's the limitation I encountered?
  • What are explanations for the error messages? They don't look very self-explaining.

UPD: Tried the same thing on Debian 7. ulimit -n <limit> works as expected and I can establish as much connections as is set by ulimit. So, it looks like an OS X specific problem.

Ivan Velichko
  • 6,348
  • 6
  • 44
  • 90
  • Most likely file descriptors. – mdorman Sep 16 '15 at 12:13
  • @mdorman, yeah, it looks like it. But even if I do `ulimit -n 10000` before starting my code, output is the same. Except `NOFILE: (10000, 10000)` line, of course. – Ivan Velichko Sep 16 '15 at 12:15
  • You should try running the same `ulimit` command, only with the `-S` parameter, i.e. `ulimit -S -n 10000`. Try it and tell us if it worked. – amito Sep 16 '15 at 14:46
  • @amito, it's not working even with `-S`. I think `resource.getrlimit(resource.RLIMIT_NOFILE)` shows real value of this limit within the test script. – Ivan Velichko Sep 16 '15 at 16:14

0 Answers0