2

Over the last couple of years, I have done a fair amount of work on Amazon SWF, but the following points are still unclear to me and I am not able to find any straight forward answers on any forums yet.

These are pretty basic requirements I suppose, sure others might have come across too. Would be great if someone can clarify these.

  1. Is there a simple way to return a workflow execution result (maybe just something as simple as boolean) back to workflow starter?
  2. Is there a way to catch Activity timeout exception, so that we can do run customised actions in such scenarios?
  3. Why doesn't WorkflowExecutionHistory contains Activities, why just Events?
  4. Why there is no simple way of restarting a workflow from the point it failed?

I am considering to use SWF for more business processes at my workplace, but these limitations/doubts are holding me back!

FINAL WORKING SOLUTION

public class ReturnResultActivityImpl implements ReturnResultActivity {

    SettableFuture future;

    public ReturnResultActivityImpl() {
    }

    public ReturnResultActivityImpl(SettableFuture future) {

        this.future = future;
    }

    public void returnResult(WorkflowResult workflowResult) {

        System.out.print("Marking future as Completed");
        future.set(workflowResult);
    }

}

public class WorkflowResult {

    public WorkflowResult(boolean s, String n) {
        this.success = s;
        this.note = n;
    }

    private boolean success;
    private String note;
}

public class WorkflowStarter {

    @Autowired
    ReturnResultActivityClient returnResultActivityClient;

    @Autowired
    DummyWorkflowClientExternalFactory dummyWorkflowClientExternalFactory;

    @Autowired
    AmazonSimpleWorkflowClient swfClient;

    String domain = "test-domain;
    boolean isRegister = true;
    int days = 7;
    int terminationTimeoutSeconds = 5000;
    int threadPollCount = 2;
    int taskExecutorThreadCount = 4;

    public String testWorkflow() throws Exception {

        SettableFuture<WorkflowResult> workflowResultFuture = SettableFuture.create();
        String taskListName = "testTaskList-" + RandomStringUtils.randomAlphabetic(8);

        ReturnResultActivity activity = new ReturnResultActivityImpl(workflowResultFuture);
        SpringActivityWorker activityWorker = buildReturnResultActivityWorker(taskListName, Arrays.asList(activity));

        DummyWorkflowClientExternalFactory factory = new DummyWorkflowClientExternalFactoryImpl(swfClient, domain);
        factory.getClient().doSomething(taskListName)

        WorkflowResult result = workflowResultSettableFuture.get(20, TimeUnit.SECONDS);
        return "Call result note - " + result.getNote();
    }

    public SpringActivityWorker buildReturnResultActivityWorker(String taskListName, List activityImplementations)

            throws Exception {

        return setupActivityWorker(swfClient, domain, taskListName, isRegister, days, activityImplementations,
                terminationTimeoutSeconds, threadPollCount, taskExecutorThreadCount);
    }
}

public class Workflow {

    @Autowired
    private DummyActivityClient dummyActivityClient;

    @Autowired
    private ReturnResultActivityClient returnResultActivityClient;

    @Override
    public void doSomething(final String resultActivityTaskListName) {

        Promise<Void> activityPromise = dummyActivityClient.dummyActivity();
        returnResult(resultActivityTaskListName, activityPromise);
    }

    @Asynchronous
    private void returnResult(final String taskListname, Promise waitFor) {

        ActivitySchedulingOptions schedulingOptions = new ActivitySchedulingOptions();

        schedulingOptions.setTaskList(taskListname);
        WorkflowResult result = new WorkflowResult(true,"All successful");

        returnResultActivityClient.returnResult(result, schedulingOptions);
    }
}
Dev
  • 195
  • 10

1 Answers1

2
  1. The standard pattern is to host a special activity in the workflow starter process that is used to deliver the result. Use a process specific task list to make sure that it is routed to a correct instance of the starter. Here are the steps to implement it:

    • Define an activity to receive the result. For example "returnResultActivity". Make this activity implementation to complete the Future passed to its constructor upon execution.
    • When the workflow is started it receives "resultActivityTaskList" as an input argument. At the end the workflow calls this activity with a workflow result. The activity is scheduled on the passed task list.
    • The workflow starter creates an ActivityWorker and an instance of a Future. Then it creates an instance of "returnResultActivity" with that future as a constructor parameter.
    • Then it registers the activity instance with the activity worker and configures it to poll on a randomly generated task list name. Then it calls "start workflow execution" passing the generated task list name as an input argument.
    • Then it wait on the Future to complete. The future.get() is going to return the workflow result.
  2. Yes, if you are using the AWS Flow Framework a timeout exception is thrown when activity is timed out. If you are not using the Flow framework than you are making your life 100 times harder. BTW the workflow timeout is thrown into a parent workflow as a timeout exception as well. It is not possible to catch a workflow timeout exception from within the timing out instance itself. In this case it is recommended to not rely on workflow timeout, but just create a timer that would fire and notify workflow logic that some business event has timed out.

  3. Because a single activity execution has multiple events associated to it. It should be pretty easy to write code that converts history to whatever representation of activities you like. Such code would just match the events that relate to each activities. Each event always has a reference to the related events, so it is easy to roll them up into higher level representation.

  4. Unfortunately there is no easy answer to this one. Ideally SWF would support restarting workflow by copying its history up to the failure point. But it is not supported. I personally believe that workflow should be written in a way that it never fails but always deals with failures without failing. Obviously it doesn't work in case of failures due to unexpected conditions. In this case writing workflow in a way that it can be restarted from the beginning is the simplest approach.

Maxim Fateev
  • 6,458
  • 3
  • 20
  • 35
  • Thanks Maxim. 1. Can you please elaborate a bit more? Is there any example of your suggested approach? 2. Yes, I am already using Flow. As I was not able to catch Workflow timeout exception, I assumed activity timeout won't work either, but will give it a go. 3. Looking at the list of History Events that I got back, I couldn't really think of a simple way of converting those to activities with status. Is there any example of your suggested approach? 4. Would have been really useful to have it, but I guess I can handle it my own way, atleast for now. – Dev Jan 12 '17 at 11:29
  • Ok, sounds a bit complex but trying to implement your suggestion. Couple of things I am not sure about...... Given resultActivityTaskList, how would the workflow get hold of the activity class? And in workflow starter, how would I get to know when the Future object is ready. Do I have to do something like while(!future.isDone()) {Thread.sleep(10);} If yes, is this the best and most efficient way? – Dev Jan 13 '17 at 14:50
  • Workflow invokes the activity through the generated client (see Activity Clients from http://docs.aws.amazon.com/amazonswf/latest/awsflowguide/clients.html). Java future.get() blocks until the future is ready, so polling is not necessary. The simplest is to use SettableFuture to make it ready: https://google.github.io/guava/releases/snapshot/api/docs/com/google/common/util/concurrent/SettableFuture.html. Another option is to use blockingQueue instead of Future to pass value from activity implementation to the starter thread. – Maxim Fateev Jan 13 '17 at 20:51
  • I will try using SettableFuture, that sounds like a better option. Another query - how can we add a Constructor for activity class that accepts a parameter. Never did that before as we use auto generated clients for activity classes. – Dev Jan 16 '17 at 15:39
  • It think you are confusing activity implementation with generated activity client. The activity implementation is a class written by you. It just needs to implement an interface that is annotated with @Activities. So you can add any parameters to its constructor as you also create instances of activity implementations before registering them with an ActivityWorker. – Maxim Fateev Jan 17 '17 at 03:24
  • If you only could make it publicly available in a form of an example :). – Maxim Fateev Jan 17 '17 at 19:18
  • Added sample code, tried to make it as informative and clear as possible! :) – Dev Jan 18 '17 at 16:50
  • I believe activity implementation should set settable on execution. Your sample doesn't have such code. – Maxim Fateev Jan 18 '17 at 18:41
  • Well spotted..... missed out on copying returnResult() method, fixed now, thanks! – Dev Jan 18 '17 at 22:36
  • Just realised that the dynamic task lists that I created as per the above discussions remain alive forever and keep accumulating the threads, ultimately breaking the app :( Not sure if there is any way to terminate a tasklist ? – Dev May 17 '17 at 16:09
  • Just create a single dynamic task list per process and dispatch activity invocations from it to an appropriate future. – Maxim Fateev May 17 '17 at 16:54
  • It is possible to stop polling on a task list by calling worker.shutdown, but in this case I would recommend a single shared worker per process. – Maxim Fateev May 17 '17 at 18:02
  • If by process you mean single workflow execution, thats what I am doing already i.e. creating dynamic taskList per workflow execution. Trying to figure out how I can shutdown the specific worker. Thanks – Dev May 17 '17 at 19:28
  • No, I mean a process from the operating system point of view. There is no reason to create a taskList per workflow execution. One taskList per process instance is enough to route the tasks to it. – Maxim Fateev May 20 '17 at 17:01