5

I'm trying to use an external process which reads the STDIN, and writes to STDOUT.

I want to write the equivalent of this in Elixir, without using an external library or wrapper script:

$ echo foo | nkf
foo

i.e. send data to nkf on stdin, and get the converted result back from nkf's stdout, knowing that it has finished processing the stream.

I was trying to do this with ports, but the problem is a single sent message can be returned in multiple received messages, so there's no way to tell when the end of the message has been reached (simplified example, "foo" is a whole file in reality):

iex(1)> port = Port.open({:spawn, "nkf -u"}, [:binary])
#Port<0.7>
iex(2)> Port.command(port, "foo")
true
iex(3)> flush
{#Port<0.7>, {:data, "fo"}}
{#Port<0.7>, {:data, "o"}}
:ok

How can I get the same bash pipe behaviour with Ports in Elixir?

Adam Millerchip
  • 20,844
  • 5
  • 51
  • 74
  • 1
    is there a problem with using System.cmd/3? – Daniel Dec 17 '22 at 13:23
  • What would be the issue with concatenating the data? – Aleksei Matiushkin Dec 17 '22 at 18:30
  • The [Port documentation](https://hexdocs.pm/elixir/Port.html) seems to cover this: you send the port a `:close` message, and it sends you a `:closed` message back when the process is exited. – David Maze Dec 18 '22 at 11:53
  • @Daniel did you try it? – Adam Millerchip Dec 19 '22 at 02:00
  • 1
    @AlekseiMatiushkin the problem is that I don't know when the end of the input is. – Adam Millerchip Dec 19 '22 at 02:00
  • @DavidMaze I'll have another look, but in my previous attempts closing closed the stream, which caused a broken pipe. – Adam Millerchip Dec 19 '22 at 02:01
  • It looks like this bash behaviour is just not possible in the beam right now without extra help, although I'd like to believe otherwise! Here's an 11-year-old related Erlang question: https://stackoverflow.com/questions/6570766/on-external-port-how-to-only-close-the-output-and-wait-for-exit-status – Adam Millerchip Dec 19 '22 at 02:02
  • If what you want to do can be accomplished with C, C++, or Rust, another option is to write it in one of those languages and wrap it with a NIF. Rust makes this easy using a library called Rustler: https://github.com/rusterlium/rustler – Jan Dec 19 '22 at 21:19
  • @Jan yes that is also one of the suggestions on the linked question - however this question is for doing it using only the standard library. – Adam Millerchip Dec 20 '22 at 06:21
  • 2
    A related question with some possible solutions in the comments: https://stackoverflow.com/questions/76398478/how-to-use-a-port-synchronously-in-erlang?noredirect=1#comment134760832_76398478 – kinkou Jun 07 '23 at 16:58

1 Answers1

2

The beam does not currently provide a way to close the stream to the process and wait for the stream from the process to finish sending. Using ports, closing the port will also close the stream from the external process, even if the process has not finished sending data. Due to this, it is not currently possible to do using only built-in features - it's necessary to enlist the help of external tools like porcelain or erlexec.

Adam Millerchip
  • 20,844
  • 5
  • 51
  • 74