0

I'm trying to setup task queuing with rabbitMQ, coupled to Node.js & React.js. I'm having trouble understanding how the task management actually works, and I cannot find a good example online.

I'm trying to send a task (like generate a lot of images on a user click) and send it to a queue so it does not block user navigation. Could anyone try to guide me through the process ?

I have my rabbitMQ server up and running, and am able to send/receive messages. I'm just having trouble converting this to a task management tool (like sending/receiving task-related data). Any help/examples are welcome here!

DrunkDevKek
  • 469
  • 4
  • 17

1 Answers1

2

Here is an example about how The Grid are "Handling resource-intensive tasks with work queues (task queues) in RabbitMQ"; where all computationally intensive work at The Grid (such as image analysing, and image processing) are off-loaded as tasks/jobs in RabbitMQ. Instead of having a web server waiting for a result immediately, it is free to keep processing other requests. RabbitMQ task queues are also used to distribute time-consuming tasks among multiple workers, and the main idea behind using task queues (for them) is to avoid doing a resource-intensive task immediately and having to wait for it to complete. A task can also be schedule to be done later.

Another example is the architecture behind CloudAMQP. It is built upon multiple small microservices, where RabbitMQ is used as messaging system. RabbitMQ is responsible for distributing events/tasks to the services that listen for them - where you have the option to send a message without having to know if another service is able to handle it immediately or not. Tasks can simply wait in the queue until the responsible service is ready.

Lovisa Johansson
  • 9,721
  • 1
  • 16
  • 20
  • Thank you for your answer. Though I've learned some things reading your links (good infos by the way!), my main concern is not quite understanding how the process works, but rather actually coding it. Let's say I have my `generateImages(args)` function, where do I call it? What message should I send (is it the args taken by the function, the type of action I want to execute, etc.), and when? I find the official tutorials/documentation lacking concrete examples. It's more of a "how can I achieve this" problem, rather than "how does this work". I don't know if that was clear enough. – DrunkDevKek Jul 24 '17 at 08:49
  • Sorry, it seems that I misunderstood your question then. And I can't help you regarding React.js. Hope you get some more answers! :) – Lovisa Johansson Jul 24 '17 at 08:56
  • It's more of a Node.js issue actually, because Rabbitmq is a server-to-server communication tool. Thank you for trying though my good man! – DrunkDevKek Jul 24 '17 at 09:03
  • The consumer takes the task/message from the queue. The message should include the task, "start jobb message" and arguments - or you can have queue for that specific task, like the queue "account.create" in the example linked above. A message could be a simple json for example. generateImages(args) is than called by the consumer with specified arguments. – Lovisa Johansson Jul 24 '17 at 09:08
  • Here is an example with Node.js https://www.cloudamqp.com/blog/2015-05-19-part2-2-rabbitmq-for-beginners_example-and-sample-code-node-js.html, where ch.consume sets up a consumer with a callback to be invoked with each message it receives. The function called for each message is called "processMsg". – Lovisa Johansson Jul 24 '17 at 09:12