Created a script to test against a server (it's capable of over 100K concurrent TCP sockets). Affter it reaches 65536, the client stopped attempting to set up connections. Wonder why.
use AnyEvent;
use EV;
use AnyEvent::Handle;
use IO::Socket::UNIX;
use Time::HiRes qw( gettimeofday );
use Data::Dumper;
my $count = 0;
my @hds = ();
my $cv = AnyEvent->condvar;
my $ips = ["192.168.1.11", "192.168.1.12","192.168.1.13","192.168.1.14"];
my $i;
my $j = 0;
for ($i=0; $i<1000; $i++) {
kickoff();
}
$cv->recv;
sub kickoff {
my $hdl = new AnyEvent::Handle
connect => ["192.168.1.1", 80],
on_error => sub {
my ($hdl, $fatal, $msg) = @_;
print "error when count=$count, $!\n";
$hdl->destroy;
$cv->send;
},
on_connect => sub {
$count ++;
if ($count > 65000) {
print "$count\n";
} elsif (($count % 1000) == 0) {
print "$count\n";
}
if ($count >= 100000) { return; }
kickoff();
},
on_prepare => sub {
my $hdl = shift;
#print Dumper($hd); exit;
mybind($hdl->{fh}, $ips->[$j%4]);
$j ++;
},
on_read => ::handleData;
push @hds, $hdl;
}
sub getTS {
my ($seconds, $microseconds) = gettimeofday;
return $seconds + (0.0+ $microseconds)/1000000.0;
}
sub mybind {
my ($sock, $ip) = @_;
my $local_host=inet_aton($ip) or return undef;
my $local = sockaddr_in(0, $local_host);
bind($sock,$local) or print 'Couldn\'t bind to local\n';
}
For the record, I ran it as root on Ubuntu 14.04 (linux 3.19.0) and ulimit -a
shows 100000.
Any idea why?
UPDATE1
Some update based on the comments. Thanks guys.
- When I use only one IP, I can easily go to 27000 concurrent TCP connections. The port range (for each host) should allow 100,000 connections on 4 hosts.
- I have
fs.nr_open = 1048576
but don't have config for net.ipv4.netfilter.ip_conntrack_max - The max number "65536" can mean many things: 1) max number of ports on a host 2) the number of numbers in a 2-byte short integer. So it could be related. Wonder where I can redefine the limit or if it's possible to redefine it.
- When I reach the number 65536, it stopped attempting to create new connections.