Questions tagged [message-queue]

"Message queue" is a design pattern or software engineering component that defines discipline or API for communication between two or more interrelated processes or systems. The message queue enforces asynchronous processing and loose coupling. Depending on the implementation they may also provide delivery and order guarantees of the messages. Message processing guarantees are explicitly excluded from the design.

Message queue design pattern introduces an intermediary between the producer of messages and its consumer(s). This intermediary - the queue - accepts the messages and stores them until a consumer retrieves them or until some other event (e.g. expiration time). In general the producer does not know or care where, how, and when the messages in the queue will be consumed.

Software components such as MSMQ or IBM MQ or Tibco (usually called Queue Managers) provide queue storage and features for manipulation of messages in queues. These manipulations include: expiration, persistence, notification of expired messages (dead-letter queues), message cloning, queue security, distribution (between multiple Queue Managers).

This tag is best used for cross-platform message queuing questions. For questions about specific products, use or include the tag for that product:

3929 questions
57
votes
4 answers

In which domains are message oriented middleware like AMQP useful?

What problem do MOM (Message Oriented Middleware) solve? Scalability? Integration? In which domain are they typically used and in which domains are they typically not used? For example, say, is Google using such solution for it's main search engine…
cocotwo
  • 1,285
  • 2
  • 13
  • 19
54
votes
4 answers

Publish/Subscribe vs Producer/Consumer?

From messaging point of view with systems like Kafka, RabbitMQ, HornetQ, ActiveMQ, etc. is there a difference between pub/sub and prod/con or are they used interchangeably?
user432024
  • 4,392
  • 8
  • 49
  • 85
53
votes
3 answers

Messaging Confusion: Pub/Sub vs Multicast vs Fan Out

I've been evaluating messaging technologies for my company but I've become very confused by the conceptual differences between a few terms: Pub/Sub vs Multicast vs Fan Out I am working with the following definitions: Pub/Sub has publishers…
ghostJago
  • 3,381
  • 5
  • 36
  • 51
51
votes
2 answers

Celery unable to use redis

Trying to start Celery first time but issues error as below, i have installed redis and its starting fine , but still somehow django seems to have issues with it , File "", line 848, in exec_module File…
Atif Shafi
  • 954
  • 1
  • 11
  • 26
49
votes
2 answers

Message queues vs sockets

I don't have much of a socket programming experience but I tried read a little about it. I am quite familiar with MDB and messaging queues. Someone has told me that queue(e.g. MDB) is "Not much more than a direct socket connection". Can someone…
Sara
  • 2,417
  • 7
  • 35
  • 52
48
votes
4 answers

Competing Consumer on Redis Pub/Sub supported?

I have 2 services. Both of them need subscribe to the same channel. The 2 services are load balanced. Each service runs on multiple servers. So how can I be sure only 1 instance of each service consume the message of that channel. Is this supported…
jordan
  • 711
  • 1
  • 8
  • 17
46
votes
5 answers

Message queue and Message broker differences

so I've been trying to understand what are the differences between message queues and message brokers and why should you use one over the other. So from what I've understand. MESSAGE QUEUE helps with inter process communications, but it is limited…
user9124444
45
votes
5 answers

Low-latency, large-scale message queuing

I'm going through a bit of a re-think of large-scale multiplayer games in the age of Facebook applications and cloud computing. Suppose I were to build something on top of existing open protocols, and I want to serve 1,000,000 simultaneous players,…
Jon Watte
  • 6,579
  • 4
  • 53
  • 63
44
votes
8 answers

Advice on Python/Django and message queues

I have an application in Django, that needs to send a large number of emails to users in various use cases. I don't want to handle this synchronously within the application for obvious reasons. Has anyone any recommendations for a message queuing…
Andy Hume
  • 40,474
  • 10
  • 47
  • 58
43
votes
3 answers

What is the difference between a channel adapter and a messaging gateway pattern?

No matter how much I read about those two patterns I just can't see the difference.
Pillblast
  • 1,171
  • 6
  • 18
  • 25
42
votes
2 answers

How does consumer rebalancing work in Kafka?

When a new consumer/brorker is added or goes down, Kafka triggers a rebalance operation. Is Kafka Rebalancing a blocking operation. Are Kafka consumers blocked while a rebalancing operation is in progress?
java_geek
  • 17,585
  • 30
  • 91
  • 113
41
votes
8 answers

How can I check whether a RabbitMQ message queue exists or not?

How can I check whether a message Queue already exists or not? I have 2 different applications, one creating a queue and the other reading from that queue. So if I run the Client which reads from the queue first, than it crashes. So to avoid that i…
Jigar Sheth
  • 586
  • 2
  • 5
  • 21
39
votes
7 answers

Message Queues in Ruby on Rails

What message queues are people using for their Rails apps and what was the driving force behind the decision to choose it. Does the latest Twitter publicity over their in house queue Starling falling down affect any existing design decisions. I am…
nitecoder
  • 5,496
  • 1
  • 28
  • 35
38
votes
5 answers

Message Queue vs Task Queue difference

I wonder what is the difference between them. Are they describing the same thing? Is Google App Engine Service Task Queue is an implementation of Message Queue?
Rudy
  • 7,008
  • 12
  • 50
  • 85
37
votes
5 answers

How does JMS Receive work internally?

I've been researching various communication technologies/architectures/patterns/implementations (read: buzzwords) including Web Services (WCF, Axis2), ESBs, SOA, and wanted to know more about JMS with regards to messaging. Conceptually, JMS sounds…
Travis
  • 2,654
  • 4
  • 26
  • 46