How do you make the output of a `readfile` function into source for conduit?

Question

I am opening some .txt file via:

main :: IO ()
main = do
  xxs  <- TIO.readFile pathToFile
  return ()

The .txt file is of form

str_1 \n str_2 \n ... str_m

And I would like to make xxs into a source so that it might look like:

sourceList [str_1, str_2, ..., str_m]

Does the conduit API provide a way to do it without doing some string manipulation on xxs first, thus making it into the form [str_1, str_2, ..., str_m]?

What about using `lines` to break up `xxs` into a list of lines? See [Breaking into lines and words](https://hackage.haskell.org/package/text-1.2.2.1/docs/Data-Text.html#g:19) — ErikR, Aug 10 '16 at 03:15
That is neat, just to check the list of strings is lazily generated by `lines` right? So the whole list never exists in memory, each line is output and immediately consumed by sourceList, and any downstream conduits ... — xiaolingxiao, Aug 10 '16 at 03:23
Are you looking only for answers using “conduit” or as the tags imply are you also open to answers using “pipes”? — R B, Aug 10 '16 at 03:47
If `TIO` refers to Data.Text.Lazy.IO, then yes. If `TIO` refers to Data.Text.IO then `readFile` will read the entire file into memory. In either case `lines` will still construct the list of lines lazily will allows for the possibility that only one line is resident in memory at a time. — ErikR, Aug 10 '16 at 03:48
@RowanBlush Open to using pipes, but strongly biased to using conduit. I figured people who used pipes have also used conduit so I was trying to cast a wider net. — xiaolingxiao, Aug 10 '16 at 03:48
Perhaps you are looking for `lines` from Data.Conduit.Text: https://hackage.haskell.org/package/conduit-1.0.2/docs/Data-Conduit-Text.html#v:lines — ErikR, Aug 10 '16 at 03:52
Why do you want to make it like a list ? Just write your won conduit function to stream it without memory leak: http://stackoverflow.com/a/38864188/1651941 — Sibi, Aug 10 '16 at 04:28

score 5 · Accepted Answer · answered Aug 10 '16 at 04:33

How do you make the output of a readfile function into source for conduit?

The source function for reading files already exists in the package conduit-extra in the form of sourceFile. You can also see various other combinators in that module like conduitFile, etc.

How do you make the output of a `readfile` function into source for conduit?

1 Answers1