Yaws - socket_closed_remotely

Question

Hello fellow Erlangers :)

Just another Erlang enthusiast playing with the language here. I have a very simple YAWS app module which works fine when accessed by single clients. However, when I try to spawn multiple concurrent clients, some of those clients start receiving errors (even if the number of those clients is very low, say 10). Any idea what can be causing it?

Illustrating code:

App Module

out(Arg) ->
  io:format("got something!\n"),
  Method = extract_method(Arg),
  handle(Method, Arg).

Client

client(SenderPID) ->
case httpc:request(
  put,
  {
    "http://localhost:8080/storageunit",
    [],
    "application/x-www-form-urlencoded",
    ""
  }, [], []) of
    { error, Reason } -> io:format("Server responded with an error: ~p.\n", [Reason]);
    { ok, _ } -> ok
  end,
  SenderPID ! 'FINISHED'.

client_spawner(_SenderPID, 0) -> io:format("Done.\n");
client_spawner(SenderPID, Times) ->
  spawn(concurrent, client, [SenderPID]),
  client_spawner(SenderPID, Times - 1).

The errors I'm getting:

Server responded with an error: socket_closed_remotely.

The last piece of info is:

10 concurrent clients - from 2 to 5 errors on average
100-100000 concurrent clients - 50% errors on average

I thought my handling code causes it, but in scenarios where the clients receive httpc errors, the server doesn't even react with a "got something!".

I'm sure I'm missing something trivial, can you help?

Regards, Piotr

It's hard to answer this without more details of what `handle/2` is doing on the server side. I recommend checking Yaws logs to see if there's any relevant there. Also, make sure your Yaws listen backlog is set high enough for what you're trying, and make sure maximum numbers of file descriptors for both client and server are set appropriately. — Steve Vinoski, Feb 11 '17 at 17:37
Thank you. See, the problem is handle/2 does not even get called. I read somewhere on github (some old issue in some repo) that it could be caused by the profile used by inets. I plan to explore that a bit. — Piotr Justyna, Feb 11 '17 at 23:09
Yes, just confirmed. Even when I completely rule the handle/2 out of the equation, I'm still getting the same problem. — Piotr Justyna, Feb 11 '17 at 23:19
Is Yaws receiving anything at all, or this is just a client-side issue? — Steve Vinoski, Feb 11 '17 at 23:50
It looks like a client-side issue. Yaws isn't getting anything at all. — Piotr Justyna, Feb 12 '17 at 11:17
OK, I have an interesting update: running the same client code against google.com yields expected results (i.e. all spawned clients run the httpc request successfully). Gist: https://gist.github.com/PiotrJustyna/2cca8798afbf9be644e1a8546a423d63 It gets really interesting. I think I will check it against my vanilla yaws locally. — Piotr Justyna, Feb 14 '17 at 09:11

score 1 · Accepted Answer · answered Feb 14 '17 at 09:48

There must have been something wrong with my initial code. I rewrote the module from scratch and it is giving me expected results. Here's the code:

-module(annoying_client).
-export([annoy/0, client/1, client_spawner/2]).

client(SenderPID) ->
  case httpc:request("http://www.google.com") of
    { error, Reason } -> io:format("Error. Reason: ~p\n", [Reason]);
    { ok, _ } -> io:format("ok\n")
  end,
  SenderPID ! 'FINISHED'.

client_spawner(_SenderPID, 0) -> io:format("Done.\n");
client_spawner(SenderPID, Times) ->
  spawn(?MODULE, client, [SenderPID]),
  client_spawner(SenderPID, Times - 1).

annoy() ->
  inets:start(),

  spawn(?MODULE, client_spawner, [self(), 100]),
  wait_for_all_jobs(1),

  inets:stop(),
  init:stop().

wait_for_all_jobs(NumberOfFinishedJobs) ->
  receive 'FINISHED' ->
    if
      NumberOfFinishedJobs < 100 ->
        wait_for_all_jobs(NumberOfFinishedJobs + 1);
      true ->
        io:format("All jobs finished.\n")
    end
  end.

It works like a charm with both:

google
local vanilla yaws

GETs and PUTs. The problems start when I cross the threshold of 2000 concurrent requests but I'm guessing it is to be expected judging by the numbers obtained locally and remotely. In case of 2000 requests, I'm getting roughly:

6 errors while requesting google.com
500 errors while requesting local yaws

The errors I'm getting:

Error. Reason: socket_closed_remotely

and

Error. Reason: {failed_connect,[{to_address,{"www.google.com",80}}, {inet,[inet],system_limit}]}

DoS protection? Incorrect configuration? Anyway, I am happy enough with my current results. Probably will experiment with other Erlang web servers just to get some performance numbers.

Thank you all.

Yaws - socket_closed_remotely

1 Answers1