Persistent Work Queue in C#

Question

Imagine I want to have a small network of worker drones possibly on separate threads and possibly on separate processes or even on different PCs. The work items are created by a central program.

I'm looking for an existing product or service that will do this all for me. I know that there is MSMQ and also MQSeries. MQSeries is too expensive. MSMQ is notoriously unreliable. A database backed system would be fine, but I don't want to own/manage/write it. I want to use someone else's work queue system.

Related Articles:

Here is a similar question, but it's advocating building a custom queue mechanism.
The queue that I like a lot is this one from Google App Engine.
http://www.codeproject.com/KB/library/DotNetMQ.aspx

+1 as I had the same question in mind. Thanks for the link to the GAE task queue. — ccellar, Aug 24 '11 at 06:27
MSMQ being unreliable? Not for me. What are you talking about? — TomTom, Mar 18 '12 at 14:26

score 13 · Answer 1 · edited Aug 18 '15 at 09:09

If you follow some guidelines you can use a database as a queue store with good success, see Using tables as Queues.

SQL Server comes with its own built-in message queuing, namely Service Broker. It allows you to avoid many of the MSMQ pitfalls when it comes to scalability, reliability and high availability and disaster recovery scenarios.

Servcie Broker is fully integrated in the database (no external store, one consistent backup/restore, one unit of failover, no need for expensive two-phase-commit DTC between message store and database, one single T-SQL API to access and program both the messages and your data) and also has some nice unique features such as transactional messaging with guaranteed Exactly-Once-In-Order delivery, correlated message locking, internal activation etc.

score 7 · Accepted Answer · edited Aug 30 '12 at 07:12

7

I have used Rabbit MQ in the past for a pet project, you could add that to your list for Queue systems.

As far as a framework to wrap the Queue's, you could take a look at http://www.nservicebus.com/ we have done a couple of basic projects here at work with that. And here's a quick example to get started: http://meisinger2.wordpress.com/2009/11/09/nservicebus-fifteen-minutes/

edited Aug 30 '12 at 07:12

Endy Tjahjono

24,120
23
83
123

answered Aug 23 '11 at 19:02

deepee1

12,878
4
30
43

2

http://masstransit-project.com/ is another options, instead of NServiceBus. Regardless, I'd suggest RabbitMQ over MSMQ any day. MSMQ is not a great MQ product, it's just included by default in Windows. Having worked with a lot of them, many of the newer ones work, ZeroMQ, ActiveMQ, etc are all also options. Rabbit is still the best I've worked with though. – Travis Sep 05 '11 at 23:26

score 1 · Answer 3 · answered Aug 23 '11 at 19:20

1

I have successfully used MassTransit in the past. It supports using MSMQ as well as RabbitMQ.

answered Aug 23 '11 at 19:20

Rex Morgan

2,979
2
21
32

Persistent Work Queue in C#

3 Answers3