8

In the Strange Loop presentation on Transducers Rich Hickey mentions at a concept in a table called 'parallel'.

enter image description here

You can easily see examples of seqs and into and channels using transducers.

Now you can work out that Observables are talking about RxJava.

My Question is What is the 'parallel' concept in Rich Hickey's transducers Strange Loop talk? Is this a list of futures, or pmap or something else?

Community
  • 1
  • 1
hawkeye
  • 34,745
  • 30
  • 150
  • 304

2 Answers2

11

There have been some thoughts about creating parallel transducible processes. This is being tracked as CLJ-1553. Currently we are not planning to address this in Clojure 1.7, but would like to do something in Clojure 1.8.

It is possible now to set up a reducer that uses a transducer as the bottom reduce phase (along with more traditional combiner fns) but ideally we would be able to leverage the "self-reducible" concept embodied by persistent vectors and maps to support transduce in parallel in a more natural way.

It is most likely right now that this would emerge as some sort of preduce function, but still much to be decided.

One problematic area is in dealing with kv forms - reducers made some choices there that are difficult or inconvenient with transducers so that needs to be worked through.

Alex Miller
  • 69,183
  • 25
  • 122
  • 167
7

The concept is simply that of performing computation in parallel. There are multiple possible implementations:

  1. clojure.core.reducers/fold, which is similar to reduce, except it should only be used with associative reduction functions and it's backed by a protocol which exploits the tree structure of various Clojure data structures to parallelize the computational effort. It's not actually transducer-friendly yet, but it is reducer-friendly and it seems that a transducer-enabled version is bound to arrive eventually.

  2. Recent releases of core.async with transducer support export a function called pipeline which parallelizes channel → channel transducer-based transformations.

Michał Marczyk
  • 83,634
  • 13
  • 201
  • 212