Questions tagged [scalaz-stream]

scalaz-stream is a streaming I/O library. The design goals are compositionality, expressiveness, resource safety, and speed. The design is meant to supersede or replace older iteratee or iteratee-style libraries.

The library supports a number of other interesting use cases:

  • Zipping and merging of streams: A streaming computations may read from multiple sources in a streaming fashion, zipping or merging their elements using a arbitrary Tee. In general, clients have a great deal of flexibility in what sort of topologies they can define--source, sinks, and effectful channels are all first-class concepts in the library.
  • Dynamic resource allocation: A streaming computation may allocate resources dynamically (for instance, reading a list of files to process from a stream built off a network socket), and the library will ensure these resources get released in the event of normal termination or when errors occur.
  • Nondeterministic and concurrent processing: A computation may read from multiple input streams simultaneously, using whichever result comes back first, and a pipeline of transformation can allow for nondeterminism and queueing at each stage.
  • Streaming parsing (UPCOMING): A separate layer handles constructing streaming parsers, for instance, for streaming JSON, XML, or binary parsing. See the roadmap for more information on this and other upcoming work.
83 questions
6
votes
1 answer

Bucketed Sink in scalaz-stream

I am trying to make a sink that would write a stream to bucketed files: when a particular condition is reached (time, size of file, etc.) is reached, the current output stream is closed and a new one is opened to a new bucket file. I checked how the…
Mortimer
  • 2,966
  • 23
  • 24
5
votes
2 answers

How can I combine two scalaz streams with a predicate selector?

I would like to combine two scalaz streams with a predicate which selects the next element from either stream. For instance, I would like this test to pass: val a = Process(1, 2, 5, 8) val b = Process(3, 4, 5, 7) choose(a, b)(_ < _).toList…
joescii
  • 6,495
  • 1
  • 27
  • 32
5
votes
1 answer

scalaz-stream how to implement `ask-then-wait-reply` tcp client

I want to implement an client app that first send an request to server then wait for its reply(similar to http) My client process may be val topic = async.topic[ByteVector] val client = topic.subscribe Here is the api trait Client { val…
jilen
  • 5,633
  • 3
  • 35
  • 84
5
votes
1 answer

What is the scalaz-stream equivalent to Play Framework's Enumerator.fromCallback

Play Framework's iteratee library defines a method Enumerator.fromCallback which allows elements to be generated based on the results of a Future: http://www.playframework.com/documentation/2.2.x/Enumerators def fromCallback[E]( retriever: () =>…
tksfz
  • 2,932
  • 1
  • 23
  • 25
4
votes
3 answers

How do I hook up scalaz-streams to reactive streams (as in reactive-streams.org)

I wanted to stream the data returned from a slick 3.0.0 query via db.stream(yourquery) through scalaz-stream. It looks like reactive-streams.org is used an API and dataflow model that different libraries implement. How do you do that with the back…
user1763729
  • 167
  • 1
  • 11
4
votes
1 answer

Why there are two implementation of network io in scalaz-stream?

They are located in scalaz.stream.tcp and scalaz.stream.nio. The tcp version treats the Connection as Process[Task, A] The nio version treats the Connection as Process[Task, Exchange[I, W]] Why do these two versions exist? Which situation are they…
jilen
  • 5,633
  • 3
  • 35
  • 84
4
votes
4 answers

Repeatly eval T => scala.concurrent.Future[T] to a Process[?, T]

I have a function get: T => scala.concurrent.Future[T] I want to iterates it like : val futs: Iterator[Future[T]] = Iterator.iterate(get(init)){ _.flatMap(prev => get(prev)) } But the type of Iterator is Future[T], it is not easy to…
jilen
  • 5,633
  • 3
  • 35
  • 84
4
votes
2 answers

How to create a circular stream?

I'm trying to create a circular process using scalaz-stream by merging one source of data with a filtered version coming from the same data source. Here is a simple example of what I have so far : val s1 = Process.emitAll(1 to 10).toSource val w =…
synapski
  • 323
  • 2
  • 12
4
votes
2 answers

Using scalaz-stream to calculate a digest

So I was wondering how I might use scalaz-stream to generate the digest of a file using java.security.MessageDigest? I would like to do this using a constant memory buffer size (for example 4KB). I think I understand how to start with reading the…
adamretter
  • 3,885
  • 2
  • 23
  • 43
4
votes
1 answer

Scalaz stream group sorted database results

I see a common pattern in my code. I have sorted results from a database and I need to emit them in a nested structure. I would like this to stream and so I want to have as few records in memory at a time. Using TravesableLike.groupBy assumes that…
rrmckinley
  • 251
  • 2
  • 7
3
votes
1 answer

Execute two fs2 tasks concurrently (non-determenistically)

With Scalaz Task I make this with scalaz.Nondeterminism.both: Nondeterminism[Task] .both( Task.now("Hello"), Task.now("world") ) or with Nondeterminism[Task].gatherUnordered(). How can I do the same thing with fs2 0.9.x version tasks?
mixel
  • 25,177
  • 13
  • 126
  • 165
3
votes
2 answers

How to implement receiveAvailable transducer in scalaz-stream

Short Version: I would like to implement a function that returns a transducer that waits for a block of values to be "emitted". The function I have in mind would have the following signature: /** * The `Process1` which awaits the next "effect" to…
jedesah
  • 2,983
  • 2
  • 17
  • 29
3
votes
2 answers

How can I speed up scalaz-stream text processing?

How can I speed up the following scalaz-stream code? Currently it takes about 5 minutes to process 70MB of text, so I am probably doing something quite wrong, since a plain scala equivalent would take a few seconds. (follow-up to another question) …
mitchus
  • 4,677
  • 3
  • 35
  • 70
3
votes
1 answer

Usage example of scalaz-stream's inflate

In the following usage example of scalaz-stream (taken from the documentation), what do I need to change if the input and/or output is a gzipped file? In other words, how do I use compress? import scalaz.stream._ import scalaz.concurrent.Task val…
mitchus
  • 4,677
  • 3
  • 35
  • 70
3
votes
1 answer

Modeling a FSM via Process1 transducer?

I have a state machine that I would like to model using a scalaz-stream Process1. The state machine models the message flow between a client and a server. A basic set of data types might be: sealed trait ServerState case object Disconnected extends…
penland365
  • 525
  • 4
  • 13