1

I want to evaluate arguments to any function in parallel transparently (without any source level changes).

For example -
    c = f(a, b) should result in:
    a and b being evaluated in parallel and then invoking of f.

One way to do this is to convert the above expression to: 

a' = future { a }
b' = future { b }
f' = lift f

(so that f: a -> b -> c becomes f: Future<a> -> Future<b> -> Future<c>) so that c' = f'(a', b')

Is this possible to do in scala?

Dave L.
  • 9,595
  • 7
  • 43
  • 69
akh
  • 101
  • 2
  • 5
  • Trying to do this *transparently* is arguably working against the grain in Scala—you'd be a lot better off aiming for *concisely* and *elegantly*. – Travis Brown Nov 08 '12 at 18:53
  • 1
    What do you mean by "transparently"? Rabid parallelism is a splendid way to create a huge amount of waste heat and take far longer than a serial program. So presumably you plan on specifying _which_ functions are eligible for parallelizing? – Rex Kerr Nov 08 '12 at 19:13
  • 1
    Just in case one thinks "oh, parallelizing everything is no big deal, we'll make up on the big stuff what we lose on the small stuff"--naively parallelized `math.max` with futures takes fully 100,000 times longer than serial `math.max`. That's pretty hard to recover from. – Rex Kerr Nov 08 '12 at 19:23

1 Answers1

4

Assuming that you're OK using Scala 2.10 (not yet released, but up to Release Candidate 2) and are happy to use an experimental feature, this should be pretty easy to achieve with Scala's macro system.

Paul Butcher
  • 10,722
  • 3
  • 40
  • 44
  • Yeah, I saw Scala's macro system earlier and it looks promising. – akh Nov 12 '12 at 22:48
  • I am only concerned about IO concurrency than parallelization. So, a and b could be some IO commands (disk/file access, remote procedure call, etc.). There are two things I want to achieve: a) thread should be blocked while waiting for IO - can be done using user level thread library e.g. gevent in python 2) schedule as many IO calls as possible (only wait for previous IO results when needed). So, I am okay with marking IO calls that need to be handled specially. But that will still require expression evaluation to be hooked into. No? – akh Nov 12 '12 at 22:53
  • So the question, I think, is why do you want this to happen transparently. Although I'm certain that you could do so with macros, what (really) does that gain you? As Rex said in his comment on your question - you'd be a lot better off aiming for something explicit, but concise and elegant. – Paul Butcher Nov 13 '12 at 08:21