amazon simple workflow- processing tasks in parallel and going to next stage only after all parallel processes of previous stage are complete

Question

I am working to create an app in which there are multiple stages- in the first stage there are multiple tasks to be executed in parallel...once all tasks of that stage have been completed, only then processing should go to next stage.

From what I read about deciders, deciders can choose from one of many possible options for the next stage.

But I want to go to next stage only when all parallel processes of current stage are complete.

Does this mean that I should set up each parallel process to invoke the next stage, and when the next stage is initialised, it should check if all parallel processes of previous stage are complete, and only then actually start processing? This will mean that all the parallel processes of first stage will invoke corresp. parallel processes of second stage, out of which only one will actually do processing (since this will be the process that finds that all processes of previous stage are complete).

Is there a better way of implementing this? So that the process of next stage is called only once?

Generally components of a system or pipeline like you're describing should be as decoupled as possible. This makes it easier to modify or replace a particular component. Having one process call the next is not very decoupled. You could use an intermediary such as a queue (or even a database table or file) as a message-passing mechanism that notifies the next component that it should run. Devil's in the details and I'm glossing over details obviously, but this is a general strategy that should almost always be considered. Abstraction and decoupling are your friends. — Girish Rao, Jun 08 '12 at 17:28

score 5 · Accepted Answer · answered Jun 22 '12 at 17:15

This can be solved using Promise objects returned by activities in flow framework.

In your decider code, have a function for stage1 which will execute multiple activities in parallel each returning a Promise object. Add all these promise objects in a List<Promise<>> and pass this List to an asynchronous method which handles stage 2.

Here is a sample decider code. stage1() executes three activities in parallel and calls asynchronous method stage2(@Wait List<Promise<Void>> promiseList). stage2 will not be started unless all the promises in promiseList are satisfied, i.e. unless all three activities in stage1 have completed.

private void stage1() {
    List<Promise<Void>> promiseList = new ArrayList<Promise<Void>>();
    Promise<Void> promise1 = activityClient.activity1();
    Promise<Void> promise2 = activityClient.activity2();
    Promise<Void> promise3 = activityClient.activity3();

    promiseList.add(promise1);
    promiseList.add(promise2);
    promiseList.add(promise3);

    stage2(promiseList);
}

@Asynchronous
private void stage2(@Wait List<Promise<Void>> promiseList) {
    activityClient.activity4();
}

amazon simple workflow- processing tasks in parallel and going to next stage only after all parallel processes of previous stage are complete

1 Answers1