Message queue IPC implementation on Linux

Question

I have a multi-processed perl system that picks up a lot of files ( may be in 10 Millions ) and works on the them one by one

Since each process must pick up exclusive files I need to have one seperate process that does the file-listing and sends the filenames to a message queue

Each worker-process will query the message-queue , get 10 jobs at a time and execute the 10 jobs until the message-queue is empty

What is the best way of implementing a message queue

[Gearman](http://p3rl.org/Gearman::XS), [TheSchwartz](http://p3rl.org/TheSchwartz), [Qudo](http://p3rl.org/Qudo) — daxim, Aug 29 '11 at 08:25

score 1 · Answer 1 · answered Aug 26 '11 at 14:17

1

You could give Queue::Beanstalk a try.

answered Aug 26 '11 at 14:17

frezik

2,316
15
13

score 1 · Answer 2 · edited Aug 29 '11 at 08:21

1

I'd vote for Net::RabbitMQ and rabbitmq

edited Aug 29 '11 at 08:21

daxim

39,270
4
65
132

answered Aug 27 '11 at 04:28

ErikR

51,541
9
73
124

Message queue IPC implementation on Linux

2 Answers2