Distributed\Parallel computing using app-engine (java api)

Question

I want to use the master-slave (worker) paradigm, to solve a problem. I have read that opening new threads manually (for example using thread pool) is not available and I need to use queue, attached code example:

class MyDeferred implements DeferredTask {
    @Override
    public void run() {
        // Do something interesting
    }
};

MyDeferred task = new MyDeferred();
// Set instance variables etc as you wish
Queue queue = QueueFactory.getDefaultQueue();
queue.add(withPayload(task));

How can I get the result of the workers (which were added to the queue)?

I need this info, in-order to solve the bigger problem.

Welcome to StackOverflow. It would be good if you could add some code of what you have tried so far in regards to fetching your result. For some tips about how to use the site please refer to the [help](http://stackoverflow.com/about) page. — Qben, Jul 04 '13 at 13:40

score 0 · Answer 1 · answered Jul 04 '13 at 15:52

You will have to write the results into the datastore.

Just as a starting point to think about it, you might pass a JobId as a parameter to the tasks, have each task write an entity with the result and the JobId, and then later query the datstore for the given JobId to get all the results.

score 0 · Accepted Answer · answered Jul 05 '13 at 06:29

0

Actually you can use threads on GAE, but there are limitations. If you need long-running threads you can use background threads, but this requires you to use backend instances.

If you opt to use task queue, then keep in mind that tasks do not "return" to caller. To aggregate results you'll need to use datastore.

answered Jul 05 '13 at 06:29

Peter Knego

79,991
11
123
154

Thanks :) Just to make sure is there any why to add extra info to the following class: [TaskHandle?](https://developers.google.com/appengine/docs/java/javadoc/com/google/appengine/api/taskqueue/TaskHandle) – user2550587 Jul 06 '13 at 07:11
To pass custom data you should implement `DeferredTask`: http://stackoverflow.com/questions/7505116/java-gae-deferredtask-example – Peter Knego Jul 06 '13 at 08:49

Distributed\Parallel computing using app-engine (java api)

2 Answers2