Mix task spawning a process does nothing

Question

I created a mix task which spawns a process. It uses Flow, reads from a stream and writes into a file defined in a consumer.

At the end of the day it just spawn some processes.

If I run it through iex it works fine as long as I leave the shell running.

But if I launch it from the command line as a mix task nothing happens, how do you leave the process opened?

  use Flow

  def run([stream]) do
    specs = [{{ProdCon,[]},[]}]
    consumer = [{{Consumer,[]},[]}]

    stream
    |> Flow.from_enumerable()
    |> Flow.through_specs(specs)
    |> Flow.into_specs(consumer)
  end

score 2 · Accepted Answer · edited Aug 01 '18 at 13:52

2

Spawn a Task doing the job and wait until it finishes with Task.yield/2:

use Flow

def run([stream]) do
  task = Task.async(fn ->

    specs = [{{ProdCon,[]},[]}]
    consumer = [{{Consumer,[]},[]}]

    stream
    |> Flow.from_enumerable()
    |> Flow.through_specs(specs)
    |> Flow.into_specs(consumer)

  end)

  case Task.yield(task, 3_600) do # wait 1 hour
    {:ok, result} -> result
    nil -> IO.puts("Failed to get a result :(")
  end
end

edited Aug 01 '18 at 13:52

lapinkoira

8,320
9
51
94

answered Aug 01 '18 at 07:48

Aleksei Matiushkin

119,336
10
100
160

mmm is not working, doesnt do anything, all the use Flow and def run are defined in another module but dont think that changes anything – lapinkoira Aug 01 '18 at 08:41
I am pretty sure the error is induced. The very same code works for me e.g. here: https://github.com/am-kantox/test_cluster_task/blob/master/lib/mix/tasks/test.cluster.ex. – Aleksei Matiushkin Aug 01 '18 at 08:57
mm might be different using Flow? – lapinkoira Aug 01 '18 at 09:04
It’s very unlikely. Try to create the dummy application and run it instead of the pure `GenServer` maybe? – Aleksei Matiushkin Aug 01 '18 at 09:04
what I dont understand is which is the difference between running it through iex (where it works) and from command line mix task – lapinkoira Aug 01 '18 at 09:49
`iex` starts a supervising process (for it’s own application,) what basically makes any running subtask to comfortably complete (app is running ⇒ ErlangVM is running ⇒ your code is running.) – Aleksei Matiushkin Aug 01 '18 at 09:51
mm should I wrap then stream |> Flow.from_enumerable() |> Flow.through_specs(specs) |> Flow.into_specs(consumer) into a worker? maybe the way is done now GenServer.stop is being called before it's done – lapinkoira Aug 01 '18 at 09:55
Indeed! Just [`Task.start_link/1`](https://hexdocs.pm/elixir/master/Task.html#start_link/1) (or even `Task.async/1`) followed by `Task.yield/2` would be actually enough. I will update the answer. – Aleksei Matiushkin Aug 01 '18 at 09:57
actually it makes sense and it works for some simple task like writing in a file, but it keeps ignoring the Flow part, I think there must be something else to make it work with a Flow – lapinkoira Aug 01 '18 at 10:37
I was able to find this for `Flow`: https://github.com/elixir-lang/flow/blob/v0.14.2/lib/flow.ex#L308-L325 so you might succeed by introducing a supervision tree and including the `Flow` into it. – Aleksei Matiushkin Aug 01 '18 at 10:41
1

I think both approachs, guard and task, would work for this, the issue is in my flow, I will accept the answer – lapinkoira Aug 01 '18 at 10:43

Mix task spawning a process does nothing

1 Answers1