0

For some reason I cannot get IPython parallel function DirectView.apply() to really parallelize a function call. DirectView.map() works as expected. Am I doing something wrong here? A working example script

import time
from datetime import datetime
from ipyparallel import Client, require

@require(time)
def wait(seconds=1):
    time.sleep(seconds)

if __name__=='__main__':
    client = Client()
    print('engine ids: {}'.format(client.ids))
    dview = client.direct_view((0, 1, 2, 3))
    dview.block = False
    print('view targets: {}'.format(dview.targets))

    print('dview.apply...')
    t0 = datetime.now()
    results = [dview.apply(wait) for i in range(4)]
    while len(results) > 0:
        results.pop(0).get()
    print('time: {}'.format(datetime.now() - t0))

    print('dview.map... ')
    t0 = datetime.now()
    results = dview.map(wait, [1]*4)
    print('time: {}'.format(datetime.now() - t0))

prints

engine ids: [0, 1, 2, 3]
view targets: [0, 1, 2, 3]
dview.apply...
time: 0:00:04.021680
dview.map... 
time: 0:00:01.013941

showing that apply clearly does not perform as I expected.

My system is Ubuntu 14.04, Python 3.4.3, IPython 4.2.0.

Thomas K
  • 39,200
  • 7
  • 84
  • 86
truhanen
  • 563
  • 6
  • 9

1 Answers1

0

Seems that I misinterpreted the documentation. Here it says that apply(f, *args, **kwargs) calls f(*args, **kwargs) on remote engines, returning the result.

If I understand correctly, this means that the function is applied on all of the engines, instead of running it once on just one of them.

truhanen
  • 563
  • 6
  • 9
  • I think it may depend on the kind of view you're using - doing it with a load-balanced view may only run it on one. I'm not sure of that, though. – Thomas K Aug 20 '16 at 22:41