2

I have a NodeJS application with the following situation: I receive a request from a user to calculate something that requires using a very complex mathematical formula. Currently, I run this entire process at that moment, figure out the value, then send it back to the user. Unfortunately, this isn't very asynchronous friendly :)

What I would like to do instead is run this entire operation in the background and once the task is complete, the worker comes back to me with whatever that computed value is, which I then send back to the user.

I've been looking at various job queues, including Kue, Celery, Resque and Beanstalk, but none of them seem to provide this feature. They work great for things like sending emails or running any kind of job that doesn't require receiving some kind of value back, just a plain was it successful or not, but there doesn't seem to be anything out there that allows for the worker to actually send a custom message back to the original producer that created the task.

Am I wrong in saying none of the above mentioned queues support this? If so, please point me to where in the docs I can learn about utilizing this feature. Otherwise, could you point me to any others that do support this functionality?

paul smith
  • 1,327
  • 4
  • 17
  • 32

2 Answers2

1

You should fetch results from a dedicated queue and then push them to user.
Use celery as tasks queue manager and long-polling or socket for return result of operations to user.
From nodeJS app you can use node-celery https://github.com/mher/node-celery or insert messages directly in the celery reserved queue.
eg

var amqp = require('amqp');
var connection = amqp.createConnection({
  host: 'localhost',
  port: 5672
});
connection.on('ready',function(){
  connection.publish('celery', {id:uuid.v4(), task:'yourapp.tasks.usefultask', args:['firstArg','secondArg'], kwargs:{}},{
      'contentType': 'application/json'
      });
    });
Emiliano M.
  • 251
  • 2
  • 4
0

You can use Celery with https://github.com/mher/node-celery

node-celery allows to queues tasks and receive the result of execution

var celery = require('node-celery'),
    client = celery.createClient({
        CELERY_BROKER_URL: 'amqp://guest:guest@localhost:5672//',
        CELERY_RESULT_BACKEND: 'amqp'
    });

client.on('connect', function() {
    client.call('factorial', [1000000], function(result) {
        console.log(result);
    });
});
mher
  • 10,508
  • 2
  • 35
  • 27
  • 1
    How those queued tasks will get processed @mher ? I saw the node-celery documentation, Its very difficult to understand what's happening. Which function in node-celery consumes the messages and processes the tasks? – Shrinath Shenoy Jul 15 '15 at 05:33