Folding a subset of a stream using pipes 4.0

Question

I'm trying to understand pipes 4.0, and want to convert some conduit code. Suppose I have a stream of Ints, and I'd like to skip the first five, then get the sum of the following 5. Using plain lists, this would be:

sum . take 5 . drop 5

In conduit, this would be:

drop 5
isolate 5 =$ fold (+) 0

Or as a complete program:

import Data.Conduit
import Data.Conduit.List (drop, isolate, fold)
import Prelude hiding (drop)

main :: IO ()
main = do
    res <- mapM_ yield [1..20] $$ do
        drop 5
        isolate 5 =$ fold (+) 0
    print res

However, I'm not quite certain how to do this with pipes.

@SassaNF The `fold` type in pipes is significantly different than the type in conduit, which is where the confusion comes from. — Michael Snoyman, Sep 25 '13 at 10:09

Ankur · Accepted Answer · 2013-09-24T12:06:21.927

5

I haven't used Pipes before, but after going through the tutorial I found it really simple:

import Pipes
import qualified Pipes.Prelude as P

nums :: Producer Int IO ()
nums = each [1..20]

process :: Producer Int IO ()
process = nums >-> (P.drop 5) >-> (P.take 5)

result :: IO Int
result = P.fold (+) 0 id process

main = result >>= print

UPDATE:

As there is no "effectful" processing in the example we can even use Identity monad as the base monad for pipe:

import Pipes
import qualified Pipes.Prelude as P
import Control.Monad.Identity

nums :: Producer Int Identity ()
nums = each [1..20]

process :: Producer Int Identity ()
process = nums >-> (P.drop 5) >-> (P.take 5)

result :: Identity Int
result = P.fold (+) 0 id process

main = print $ runIdentity result

UPDATE 1:

Below is the solution I came up with (for the gist link comment), but I feel like it can be made more elegant

fun :: Pipe Int (Int, Int) Identity ()
fun = do
  replicateM_ 5 await
  a <- replicateM 5 await
  replicateM_ 5 await
  b <- replicateM 5 await
  yield (sum a, sum b)

main = f $ runIdentity $ P.head $ nums >-> fun where
  f (Just (a,b)) = print (a,b)
  f Nothing = print "Not enough data"

edited Sep 24 '13 at 12:06

answered Sep 24 '13 at 09:02

Ankur

33,367
2
46
72

Thanks, that's what I was looking for. I tried something similar to that which didn't work, though I can't seem to find what it was. – Michael Snoyman Sep 24 '13 at 10:37
Actually, I think the thing I ended up getting really tripped up on was this slightly more complicated example: https://gist.github.com/snoyberg/6683033, but that's really a separate issue. If you have any ideas on it, let me know, otherwise I'll just open it as its own question. – Michael Snoyman Sep 24 '13 at 10:41
Note that there is a `sum` pipe you can use, too, but this is still a very good answer. – Gabriella Gonzalez Sep 24 '13 at 14:52
I understand that that answer gives the same result, but it's quite different than the gist. (1) It's reading the values into memory instead of doing a streaming fold. (2) For the case of `sum` this works fine, but if the consumer was something more complicated (e.g., zlib-compressing and saving to a file), we couldn't just pull in a `Prelude` function. (3) It behaves differently if there is not enough input. – Michael Snoyman Sep 24 '13 at 15:58

score 4 · Answer 2 · answered Sep 24 '13 at 22:18

To answer your comment, this still works in the general case. I also posted the same answer on reddit where you also asked a similar question there, but I'm duplicating the answer here:

import Pipes
import Pipes.Parse
import qualified Pipes.Prelude as P

main :: IO ()
main = do
    res <- (`evalStateT` (each [1..20])) $ do
        runEffect $ for (input >-> P.take 5) discard
        P.sum (input >-> P.take 5)
    print res

This will generalize to the more complicated cases that you had in mind.

Folding a subset of a stream using pipes 4.0

2 Answers2