10

I'm looking to know is it possible to move / merge messages from one queue to another. For example:

main-queue contains messages ['cat-1','cat-2','cat-3','cat-4','dog-1','dog-2','cat-5']

dog-queue contains messages ['dog-1, dog-2, dog-3, dog-4]

So the question is, (assuming both queues are on the same cluster, vhost) it possible to move messages from dog-queue to main-queue using rabbitmqctl ?

So at the end I'm looking to get something like:

Ideally:

main-queue : ['cat-1','cat-2','cat-3','cat-4','dog-1','dog-2','cat-5', dog-3, dog-4]

But this is ok too:

main-queue : ['cat-1','cat-2','cat-3','cat-4','dog-1','dog-2','cat-5', 'dog-1, dog-2, dog-3, dog-4]

Community
  • 1
  • 1
Vor
  • 33,215
  • 43
  • 135
  • 193

1 Answers1

22

What you are/were looking for is the 'shovel' plugin. The shovel plugin comes built into the core but you have to explicitly enable it. It's really easy to use as it does everything for you (no manually consuming/republishing to another queue).

Enable shovel plugin via cli:

sudo rabbitmq-plugins enable rabbitmq_shovel

If you manage RabbitMQ via GUI, install the shovel mgmt plugin too:

sudo rabbitmq-plugins enable rabbitmq_shovel_management

Login to the GUI and you will see Shovel Management under the Admin section. You can create shovels to move messages from any queue to another queue, even remotely hosted queues. You can delete the shovel when it's finished, or leave it there and it'll continually move the messages as they come in.

If you manage RabbitMQ via CLI, you can execute shovel directly from rabbitmqctl:

sudo rabbitmqctl set_parameter shovel cats-and-dogs \
'{"src-uri": "amqp://user:pass@host/vhost", "src-queue": "dog-queue", \
"dest-uri": "amqp://user:pass@host/vhost", "dest-queue": "main-queue"}'

Official plugin docs:

Shovel Plugin - https://www.rabbitmq.com/shovel.html
Creating Shovels - https://www.rabbitmq.com/shovel-dynamic.html

rocksfrow
  • 851
  • 8
  • 12
  • 1
    That's right. I didn't noted that it possible via shovels. Thanks for providing correct answer and sorry for incorrect answer. – pinepain Nov 10 '14 at 21:57
  • 1
    Note, you don't have to enable shovel plugin on every machine, while you can set it up even on some remote machine and have both source and destination also remote servers. – pinepain Nov 10 '14 at 22:03
  • @rocksfrow, installed `rabbitmq_management` on all nodes is not a must. – pinepain Nov 10 '14 at 22:05
  • @zaq178miami I updated my answer and removed the note about installing the management plugin on every node, although I still personally think you should the plugin on all of your nodes -- otherwise you'll potentially run into management issues after a node failure. – rocksfrow Apr 17 '15 at 19:34
  • @rocksfrow, or any one here, what will be the output of this?? – Anil Kumar Jul 23 '15 at 13:56
  • 1
    I tried this, but it is not showing any success/failure message – Anil Kumar Jul 23 '15 at 13:56
  • For future reference. I am using the plugin described by @rocksfrow I had a issue where my "Move" function with the admin GUI didn't work then I deleted the temporary shovel that was created and tried again then on the second attempt it moved the items. – John May 07 '18 at 10:07