Erlang gen_server eaddrnotavail after 16358 gen_tcp:connect calls

Question

I'm writing a server in Erlang and want to open a large number of connections. The problem is that I receive {error, eaddrnotavail} after 16358 gen_tcp:connect/3 calls. As you can see below, the server is pretty simple.

I modified my MacOS Yosemite limit; ulimit -n output is 6553600.

I start with:

erl +Q 134217727 +P 1000000 -env ERL_MAX_PORTS 40960000 -env ERTS_MAX_PORTS 40960000

The server:

-module(naive_server).
-author("Stefan Stan").

-compile(export_all).

start_server(Port) ->   
  Pid = spawn_link(fun() ->
    {ok, Listen} = gen_tcp:listen(Port, [binary, {backlog, 6553600}]),
    spawn(fun() -> acceptor(Listen, 0) end),
    timer:sleep(infinity)
  end),
  {ok, Pid}.

acceptor(ListenSocket, Nr) -> 
  {ok, Socket} = gen_tcp:accept(ListenSocket),
  io:format("Client nr ~p connected~n", [Nr]),
  spawn(fun() -> acceptor(ListenSocket,Nr+1) end),
  handle(Socket).

handle(Socket) ->
  receive
    {tcp, Socket, <<"quit", _/binary>>} ->
      gen_tcp:close(Socket);
    {tcp, Socket, Msg} ->
      gen_tcp:send(Socket, Msg),
      handle(Socket)
  end.

The client:

-module(naive_client).
-author("Stefan Stan").

%% API
-export([connect_clients/3, connect/1]).

connect(Port) ->
  gen_tcp:connect({127,0,0,1}, Port, []).

connect_clients(Number, Port, List) when is_number(Number), Number >= 0, is_number(Port), Port>0 ->
  F =
    fun() ->
      case Number of
        0 -> {ok, lists:reverse(List)};
        _ ->
          {ok, Pid} = connect(Port),
          NewList = [Pid|List],
          connect_clients(Number-1, Port, NewList)
      end
    end,
  spawn(F).

score 4 · Accepted Answer · answered Sep 18 '15 at 10:34

4

I think, you may have run out of ephemeral outgoing ports (used by your client). Can you run:

macosx$ sysctl net.inet.ip.portrange.first net.inet.ip.portrange.last.

On my machine it gives me:

net.inet.ip.portrange.first: 49152
net.inet.ip.portrange.last: 65535

The difference is 16383, which is very close to your number.

answered Sep 18 '15 at 10:34

Paweł Pikuła

86
3

Yes, that was it. Now I set the first to 1025 and the last to 65535 and I can open 64510 connections. I'm looking now for a way to open even more, if you know a way please tell me... Thanks for your time – Ștefan Stan Sep 21 '15 at 03:35
The limit here is actually the TCP protocol. When you look at the tcp header, you will see that the port number is a 16 bit integer, so you won't bee able to generate more than ~ 65535 of ports per one IP, but of course you can add another network interface with a different IP. I don't know how to add another interface/subinterface on OSX. Sometimes it is easier to just add another machine. If you manage to add an extra interface, you can specify which one to use in the `gen_tcp:connect` options using {ip, ip_address()} option. – Paweł Pikuła Sep 21 '15 at 23:27
Indeed; I moved my dev on a Oracle Linux 7 and I can open over 100K connections even if I have the params: net.inet.ip.portrange.first: 49152 net.inet.ip.portrange.last: 65535 OSX is weird >.> anyway, thanks for your time – Ștefan Stan Sep 22 '15 at 08:04

Erlang gen_server eaddrnotavail after 16358 gen_tcp:connect calls

1 Answers1