How to pipe output from an IO action into a process in haskell

Question

I want to create a process and write some text from my haskell program into the process's stdin periodically (from an IO action).

The following works correctly in GHCi but don't work correctly when built and run. In GHCi everything works perfectly and the value from the IO action is fed in periodically. When built and run however, it seems to pause for arbitrarily long periods of time when writing to stdin of the process.

I've used CreateProcess (from System.Process) to create the handle and tried hPutStrLn (bufferent set to NoBuffering -- LineBuffering didnt work either).

So I'm trying the process-streaming package and pipes but can't seem to get anything to work at all.

The real question is this: How do i create a process from haskell and write to it periodically?

Minimal example that exhibits this behavior:

import System.Process
import Data.IORef
import qualified Data.Text as T  -- from the text package
import qualified Data.Text.IO as TIO
import Control.Concurrent.Timer  -- from the timers package
import Control.Concurrent.Suspend -- from the suspend package

main = do
    (Just hin, _,_,_) <- createProcess_ "bgProcess" $
        (System.Process.proc "grep"  ["10"]) { std_in = CreatePipe }

    ref <- newIORef 0 :: IO (IORef Int)
    flip repeatedTimer (msDelay 1000) $ do
        x <- atomicModifyIORef' ref $ \x -> (x + 1, x)
        hSetBuffering hin NoBuffering
        TIO.hPutStrLn hin $ T.pack $ show x

Any help will be greatly appreciated.

danidiaz · Answer 1 · 2016-05-26T07:09:16.297

3

This is a pipes Producer that emits a sequence of numbers with a second delay:

{-# language NumDecimals #-}
import Control.Concurrent
import Pipes
import qualified Data.ByteString.Char8 as Bytes

periodic :: Producer Bytes.ByteString IO ()
periodic = go 0
    where
        go n = do
            d <- liftIO (pure (Bytes.pack (show n ++ "\n"))) -- put your IO action here
            Pipes.yield d
            liftIO (threadDelay 1e6)
            go (succ n)

And, using process-streaming, we can feed the producer to an external process like this:

import System.Process.Streaming

main :: IO ()
main = do
    executeInteractive (shell "grep 10"){ std_in = CreatePipe } (feedProducer periodic)

I used executeInteractive, which sets std_in automatically to NoBuffering.

Also, if you pipe std_out and want to process each match immediately, be sure to pass the --line-buffered option to grep (or use the stdbuf command) to ensure that matches are immediately available at the output.

edited May 26 '16 at 07:09

answered May 25 '16 at 21:59

danidiaz

26,936
4
45
95

i was hoping for an answer like this. I'll try this as soon as i can and let you know (its 4 am right now here :)) – Ajit Singh May 25 '16 at 22:21
Be sure to pass the `--line-buffered` option to `grep` if you want to ensure that each match is immediately available at the output. – danidiaz May 25 '16 at 22:45
Thank you :) this worked for me with the grep example (and for cat); but my actual application is still displaying the older behavior, which seems like something else in my code. Let me just profile my app and find out, but this seems like the answer I needed. – Ajit Singh May 26 '16 at 09:31

ErikR · Answer 2 · 2016-05-25T21:56:16.750

0

What about using threadDelay, e.g.:

import Control.Monad (forever)
import Control.Concurrent (threadDelay)
...

forever $ do
    x <- atomicModifyIORef' ref $ \x -> (x + 1, x)
    hSetBuffering hin NoBuffering
    TIO.hPutStrLn hin $ T.pack $ show x
    threadDelay 1000000  -- 1 sec

Spawn this off in another thread if you need to do other work at the same time.

You can remove he need for the IORef with:

loop h x = do 
    hSetBuffering h NoBuffering
    TIO.hPutStrLn h $ T.pack $ show x
    threadDelay 1000000
    loop h (x+1)

And, of course, you only need to do the hSetBuffering once - e.g. do it just before you enter the loop.

edited May 25 '16 at 21:56

answered May 25 '16 at 21:45

ErikR

51,541
9
73
124

threadDelay is very inaccurate as a timing mechanism which is why im using suspend (it only specifies a minimum wait). I've tried the hSetBuffering solution, but it made no difference. – Ajit Singh May 25 '16 at 22:19
well - `suspend` is implemented in terms of `threadDelay`... pretty much all timing delays in Haskell are. – ErikR May 25 '16 at 22:40
Does this address the actual question (about finding there are arbitrarily long delays when compiled) at all? It appears to contain only stylistic suggestions. – Daniel Wagner May 25 '16 at 22:54
I thought the actual questions was: `The real question is this: How do i create a process from haskell and write to it periodically?` – ErikR May 25 '16 at 22:57
@ErikR I think it addresses that question only if you identify what went wrong with the given code and provide evidence that this code does *not* go wrong in that same way. – Daniel Wagner May 25 '16 at 23:02
I actually did compile and run the code with `grep 1` instead of `grep 10`, and I didn't see any increasing delays - lines came out once a second in the 10's and the 100's. I posted it because it was a simpler (and more idiomatic) way to perform loop and it seemed to work for me. – ErikR May 25 '16 at 23:11

How to pipe output from an IO action into a process in haskell

2 Answers2