Elixir httpc error on production

Question

I want to use :httpc as my HTTP client.

I use edeliver for my release management and such. I have :inets and :httpc in my .deliver/config.exs like:

  set applications: [
    :runtime_tools,
    :inets,
    :httpc
  ]

I also added :inets to :extra_applications in mix.exs.

Here is how I use :httpc:

headers =
  if apikey,
    do: [{'Content-Type', 'application/json'}, {'apikey', to_charlist(apikey)}],
    else: [{'Content-Type', 'application/json'}]

http_options = [timeout: @timeout, connect_timeout: @timeout]
options = []

request = {
  to_charlist(url),
  headers,
  'application/json',
  to_charlist(encoded_obj)
}

:post
|> :httpc.request(request, http_options, options)
|> handle_response()

I get a lot of errors like:

=SUPERVISOR REPORT==== 6-Mar-2018::15:44:11 ===
     Supervisor: {local,httpc_handler_sup}
     Context:    child_terminated
     Reason:     {function_clause,
                     [{http_transport,close,
                          [undefined,#Port<0.21557>],
                          [{file,"http_transport.erl"},{line,346}]},
                      {gen_server,try_terminate,3,
                          [{file,"gen_server.erl"},{line,648}]},
                      {gen_server,terminate,10,
                          [{file,"gen_server.erl"},{line,833}]},
                      {proc_lib,init_p_do_apply,3,
                          [{file,"proc_lib.erl"},{line,247}]}]}
     Offender:   [{pid,<0.2596.1>},
                  {id,undefined},
                  {mfargs,{httpc_handler,start_link,undefined}},
                  {restart_type,temporary},
                  {shutdown,4000},
                  {child_type,worker}]

and also

15:44:11.743 [error] GenServer #PID<0.2595.1> terminating
** (FunctionClauseError) no function clause matching in :http_transport.close/2
    (inets) http_transport.erl:346: :http_transport.close(:undefined, #Port<0.21559>)
    (stdlib) gen_server.erl:648: :gen_server.try_terminate/3
    (stdlib) gen_server.erl:833: :gen_server.terminate/10
    (stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3
Last message: {:init_error, :error_sending, {#Reference<0.18155839.2531262466.203553>, {:error, :einval}}}

which are the same error reported differently.

This line says something that I don't get too:

15:44:11.741 [error] Bad value on output port 'tcp_inet'

I don't actually get why this happens.

I was using HTTPotion and that did not have this problem (had others though).

The thing is this works on my dev machine. It also works on a production-like VM that is on my machine too. But it throws this error when it goes on real production server.

I'm so confused!

In erlang, I can successfully make an httpc cgi post request to an apache server when the body is formatted as json, but I cannot successfully make an httpc cgi post request to an inets httpd server when the body is formatted as json. I am able to successfully make an httpc cgi post request to an inets httpd server when the body is formatted as `x-www-form-urlencoded`. Could you be experiencing something similar? — 7stud, Mar 07 '18 at 21:31
@7stud that's odd. But endpoint is not using httpd, it's kestrel. I think I need to compare dev and prod erlang version. I will update question with more info if I can. — vfsoraki, Mar 08 '18 at 10:25

7stud · Answer 1 · 2018-03-07T09:43:46.340

In erlang, there is a Request parameter variable for the httpc:request() function:

request(Method, Request, HTTPOptions, Options) ->

The type of the Request parameter variable is specified as:

Request = request()
request() = {url(), headers()}

where url() = string() and headers() = [header()], i.e a list, so the shape of the Request argument needs to be:

{string, []}

But, you are doing this:

request = {
  to_charlist(url),
  headers,
  'application/json',
  to_charlist(encoded_obj)
}

which looks like it has a shape of:

{string, [], string, string}

I have no idea why you are duplicating the Content-Type of 'application/json' outside the headers list, and it's not clear to me what to_charlist(encoded_obj) is supposed to be. I would try:

request = {
  to_charlist(url),
  headers
}

If to_charlist(encoded_obj) needs to be in an http header, then add an appropriate key/value pair to your headers list. Potential http headers here. Maybe you need:

headers = 
  if api,
     do: [{'Content-Type', 'application/json'}, 
          {'apikey', to_charlist(apikey)}, 
          {'Authorization, to_charlist(encoded_obj)} ],
     else: [{'Content-Type', 'application/json'}, 
            {'Authorization, to_charlist(encoded_obj)} ]

which could probably be better written as:

base_headers = [
    {'Content-Type', 'application/json'},
    {'Authorization', to_charlist(encoded_obj)}
]

request_headers = 
    if apikey,
        do: [ {'apikey', to_charlist(apikey)} | base_headers ],
        else: base_headers

The thing is this works on my dev machine. It also works on a production-like VM that is on my machine too. But it throws this error when it goes on real production server.

Well, then see here:

Bad value on output port 'tcp_inet'

Read the document carefully. That 4-element tuple is url, headers, content type and body in respect. Also, to_charlist is used to convert Elixir strings to Erlang strings. — vfsoraki, Mar 07 '18 at 09:48
@vfsoraki, Ah, I see. I missed the pipe: `| {url(), headers(), content_type(), body()}` — 7stud, Mar 07 '18 at 09:59

Elixir httpc error on production

1 Answers1