12

I've got a distributed system where there will be 1 SQL Server, 1-n processing servers, and 1-n data suppliers (hardware devices across the network). The data being supplied will require processing prior to going into the relational DB structure - performed by the processing servers (as windows services - .net code to parse the data, process it, and insert it into the relational structure.)
To handle the potential load and not slow down the data suppliers, I want to implement a queue, but I'm not sure I want to add the complexity of an MSMQ server to the mix. Is there a good alternative to MSMQ, such as using the DB (a flat table) as the queue? Does .NET provide any out-of-the-box support for DB queues, or is there another option for reliable queueing?
Thanks

EDIT: (29 Nov, 11:30pm)
sounds like SQL Service Broker (SSB) might do the trick.
http://www.netframeworkdev.com/windows-communication-foundation/service-broker-vs-msmq-as-reliable-queueing-mechanism-63981.shtml

EDIT: (30 Nov, 7:45am)
Found another very useful link on this subject:
http://social.msdn.microsoft.com/Forums/en-US/sqlservicebroker/thread/52687510-0852-44f3-bfcd-83610d1c1b9a
I'm also looking into the max/min size of the data that will be supplied. Of the top of their head, does anyone know the max size accommodated by MSMQ and/or SSB?
MSMQ: 4MB message size
SSB: 2GB message size

EDIT: (30 nov, 8;15am)
Great comparison between MSMQ & SSB here:
Good Strategy for Message Queuing?

Community
  • 1
  • 1
Ed Sinek
  • 4,829
  • 10
  • 53
  • 81

2 Answers2

4

I would use MSMQ, it doesnt add that much complexity, and it is so easy to backup the messages, so processing can continue even after a system restart. You could use something like SSB.

fARcRY
  • 2,338
  • 6
  • 24
  • 40
2

Out of the two options, MSMQ is actually the more simplistic. If you need the ability to re-prioritize work, or have processing agents only pick certain types of queued jobs, then you cant use MSMQ. If you dont need any of those bells and whistles, then MSMQ is cake to use with .NET.

StingyJack
  • 19,041
  • 10
  • 63
  • 122
  • 1
    You can iterate through an MSMQ queue only processing some messages: `MessageQueue.CreateCursor`, and then use `Peek` and `Receive` overloads that take a cursor. – Richard Nov 30 '10 at 16:59
  • 1
    You would be better off having separate queues for each message type if you want to pick and choose certain types of message to process. – Trevor Pilley Nov 13 '12 at 11:50
  • @Richard its the "re-prioritize" that you can't really do. – StingyJack Nov 13 '12 at 14:58