Questions tagged [message-queue]

"Message queue" is a design pattern or software engineering component that defines discipline or API for communication between two or more interrelated processes or systems. The message queue enforces asynchronous processing and loose coupling. Depending on the implementation they may also provide delivery and order guarantees of the messages. Message processing guarantees are explicitly excluded from the design.

Message queue design pattern introduces an intermediary between the producer of messages and its consumer(s). This intermediary - the queue - accepts the messages and stores them until a consumer retrieves them or until some other event (e.g. expiration time). In general the producer does not know or care where, how, and when the messages in the queue will be consumed.

Software components such as MSMQ or IBM MQ or Tibco (usually called Queue Managers) provide queue storage and features for manipulation of messages in queues. These manipulations include: expiration, persistence, notification of expired messages (dead-letter queues), message cloning, queue security, distribution (between multiple Queue Managers).

This tag is best used for cross-platform message queuing questions. For questions about specific products, use or include the tag for that product:

3929 questions
34
votes
5 answers

How to Guarantee Message delivery with Celery?

I have a python application where I want to start doing more work in the background so that it will scale better as it gets busier. In the past I have used Celery for doing normal background tasks, and this has worked well. The only difference…
Ken Cochrane
  • 75,357
  • 9
  • 52
  • 60
34
votes
3 answers

Message broker vs. MOM (Message-Oriented Middleware)

I'm a little confused as to what the difference is between a message broker e.g. RabbitMQ and Message-orientated Middleware. I can't find much info apart from what's on Wikipedia. When searching MOM I find info on AMQP which states is a protocol for…
user1577433
  • 501
  • 1
  • 6
  • 12
34
votes
7 answers

Middleware & SOA by Example

I am an inexperienced Java developer trying to wrap my head around some fundamental middleware/SOA concepts and technologies, specifically: Service-Oriented Architecture (SOA) Message-Oriented Middleware (MOM) Message Queue Apache…
IAmYourFaja
  • 55,468
  • 181
  • 466
  • 756
33
votes
4 answers

Asynchronous processing or message queues in PHP (CakePHP)

I am building a website in CakePHP that processes files uploaded though an XML-RPC API and though a web frontend. Files need to be scanned by ClamAV, thumbnails need to be generated, et cetera. All resource intensive work that takes some time for…
Sander Marechal
  • 22,978
  • 13
  • 65
  • 96
32
votes
3 answers

Difference between Apache Thrift and ZeroMQ

I understand that Apache Thrift and ZeroMQ are softwares belonging to different categories, and it is not easy to do a comparison since it is an apple to orange comparison. But I don't know why they belong to different categories. Aren't they both…
Joyce Babu
  • 19,602
  • 13
  • 62
  • 97
29
votes
14 answers

Cannot find the declaration of element 'beans'

I have the spring jars of spring-3.2.0.RC1.jar and trying to implement Apache ActiveMQ helloWorld program from tutorial given here. The xml configuration file is:
nebula
  • 3,932
  • 13
  • 53
  • 82
28
votes
3 answers

Persistent Work Queue in C#

Imagine I want to have a small network of worker drones possibly on separate threads and possibly on separate processes or even on different PCs. The work items are created by a central program. I'm looking for an existing product or service that…
101010
  • 14,866
  • 30
  • 95
  • 172
28
votes
9 answers

Message Queue Error: cannot find a formatter capable of reading message

I'm writing messages to a Message Queue in C# as follows: queue.Send(new Message("message")); I'm trying to read the messages as follows: Messages messages = queue.GetAllMessages(); foreach(Message m in messages) { String message = m.Body; //do…
macleojw
  • 4,113
  • 10
  • 43
  • 63
28
votes
2 answers

Rabbitmq- Designing a message replay service

I am trying to design a replay mechanism that will enable users to replay messages from the queues. The best design I have come up for an exchange that contains multiple queues and multiple consumers is: Create a recorder service that will: Create…
Bick
  • 17,833
  • 52
  • 146
  • 251
28
votes
1 answer

Amazon SQS message multi-delivery

I understand that to bring vast scalability and reliability, SQS does extensive parallelization of resources. It uses redundant servers for even small queues and even the messages posted to the queues are stored redundantly as multiple copies. These…
inquisitive
  • 3,549
  • 2
  • 21
  • 47
28
votes
3 answers

Getting number of messages in a RabbitMQ queue

We're using amqplib to publish/consume messages. I want to be able to read the number of messages on a queue (ideally both acknowledged and unacknowledged). This will allow me to show a nice status diagram to the admin users and detect if a certain…
Basic
  • 26,321
  • 24
  • 115
  • 201
28
votes
2 answers

RabbitMQ: messages remain "Unacknowledged"

My Java application sends messages to RabbitMQ exchange, then exchange redirects messages to binded queue. I use Springframework AMQP java plugin with RabbitMQ. The problem: message comes to queue, but it stays in "Unacknowledged" state, it never…
sunny
  • 1,887
  • 3
  • 29
  • 40
27
votes
5 answers

Mock or simulate Message Queue (JMS)

There is a message(text), which format and content i definitely know. For now,class in Java,that parses and reads this message from file,is implemented. In real world, this message will come from Message Queue. For now I should simulate, mock or…
sergionni
  • 13,290
  • 42
  • 132
  • 189
27
votes
1 answer

Calling finish() After Starting a New Activity

The first Activity that loads in my application is an initialization activity, and once complete it loads a new Activity. I want to ensure if the user presses 'Back' they go straight to the Launcher, and not the initialization screen. Side note, is…
stormin986
  • 7,672
  • 15
  • 42
  • 54
27
votes
3 answers

Kafka - How to commit offset after every message using High-Level consumer?

I'm using Kafka's high-level consumer. Because I'm using Kafka as a 'queue of transactions' for my application, I need to make absolutely sure I don't miss or re-read any messages. I have 2 questions regarding this: How do I commit the offset to…