Questions tagged [message-queue]

"Message queue" is a design pattern or software engineering component that defines discipline or API for communication between two or more interrelated processes or systems. The message queue enforces asynchronous processing and loose coupling. Depending on the implementation they may also provide delivery and order guarantees of the messages. Message processing guarantees are explicitly excluded from the design.

Message queue design pattern introduces an intermediary between the producer of messages and its consumer(s). This intermediary - the queue - accepts the messages and stores them until a consumer retrieves them or until some other event (e.g. expiration time). In general the producer does not know or care where, how, and when the messages in the queue will be consumed.

Software components such as MSMQ or IBM MQ or Tibco (usually called Queue Managers) provide queue storage and features for manipulation of messages in queues. These manipulations include: expiration, persistence, notification of expired messages (dead-letter queues), message cloning, queue security, distribution (between multiple Queue Managers).

This tag is best used for cross-platform message queuing questions. For questions about specific products, use or include the tag for that product:

3929 questions
1
vote
1 answer

C# BigData Queue

We´ve got an REST server which runs on a seperate machine than the main application server. Now we want to shift the data amongst other things from the REST server to the main-application server, also we want to send some messages from main-server…
Felix Arnold
  • 839
  • 7
  • 35
1
vote
1 answer

Scheduling and handling a time consuming function to run at given date

I have a python function that should run a given date and time, which grab a video from an url, convert it to a given format and upload it to a storage. This scheduled function relies on 3 other functions synchronously (which have Separation of…
1
vote
1 answer

Nginx Requests to RabbitMQ

Is it possible for Nginx to put incoming HTTP requests into a "requests" queue in RabbitMQ and listen to responses on a "response" queue? This way, backend crashes could be made invisible to the API client by having a retry mechanism for failed…
user690545
  • 23
  • 3
1
vote
1 answer

aws worker queue with long idle and sudden burst

I'm trying to implement a worker queue to handle burst of messages. Every few days/weeks I get a burst of ~5,000 messages that need to be processed in a reasonable time (each message can be proceeded in about 1-2 min). Ideally if I could run 5,000…
1
vote
0 answers

Why do I get this error when connecting to IBM MQ?

I want to connect to IBM MQ but I get the error IBM.WMQ.MQException: 'MQRC_NOT_AUTHORIZED' on the line _queueManager = new MQQueueManager(managerName, connectionParameters); I have access to the queue manager. This is the class…
littleO
  • 33
  • 6
1
vote
0 answers

Is there a redis pub/sub replacement option, with high availability and redundancy, or, probably p2p messaging?

I have an app with hundreds of horizontally scaled servers which uses redis pub/sub, and it works just fine. The redis server is a central point of failure. Whenever redis fails (well, it happens sometimes), our application falls into inconsistent…
roman
  • 892
  • 9
  • 26
1
vote
1 answer

Azure Function QueueTrigger and int message

I want to save int value to queue message and then get it on Azure Function QueueTrigger. I save by the following way: int deviceId = -1; await queue.AddMessageAsync(new CloudQueueMessage(deviceId.ToString())); and then listen…
Oleg Sh
  • 8,496
  • 17
  • 89
  • 159
1
vote
1 answer

How to pass a filename from a second instance of a program to the main instance?

Let say an instance of my program is running. It has 'File1.My' open (whatever that is). Now the user double clicks in WinExplorer 'File2.My'. The 'My' extension is associated with my program so, a second instance will start. I use WMCOPYDATA to…
Gabriel
  • 20,797
  • 27
  • 159
  • 293
1
vote
3 answers

How to fetch data effectively from kestrel queue

For some reason we plan to use kestrel queue in our project. We do some demons, the main problem is how to to fetch data from queue with low CPU utilization and effectively. The way we implemented to fetch is if we failed to fetch data from queue…
Danyun Liu
  • 3,072
  • 24
  • 22
1
vote
1 answer

Subscribe specific message in RMQ message queue

I have a RM queue test-queue. In there I need to handle 3 separate messages (message-1, message-2, message-3) for 3 separate processes in 3 separate services. I use @RabbitListener like below to access the message @RabbitListener(queues =…
DeC
  • 2,226
  • 22
  • 42
1
vote
1 answer

Making message queues non-persistent in Linux

This may be a completely newbie question about message Queues, but we are having some issues with them being persistent across the Linux system. We are running Fedora 14 and using message queues to pass data to and from threads. The problem is we…
user626201
  • 1,623
  • 3
  • 19
  • 36
1
vote
1 answer

boost::interprocess message queue not removing

Using boost::interprocess message queue, I ran into a problem with removing the queue, was hoping someone could provide an explanation for the behavior, or correct my misunderstanding. I'm on a Linux RHEL7 box. I have two processes, one that creates…
JackR
  • 141
  • 10
1
vote
0 answers

rqscheduler docker is stopping with timemismatch error

I have created 3 dockers in same network redis queue rq scheduler Python based docker Error is coming when redis is trying to schedule the task on scheduler. docker ps output b18b7d21894f redis "docker-entrypoint.s…" …
1
vote
1 answer

Microservice Design with SignalR

We have built a microservice architecture but have run into an issue where the messages going on to the bus are too large. (Discovered since moving to Azure Service bus as this only allows 256KB compared to RabbitMQ 4MB) We have a design as the…
Chris
  • 43
  • 5
1
vote
3 answers

responsively checking two queues without pegging CPU

I have a thread pool system which uses message passing to organize events, and I am also using the Windows API which also does a bit of message passing. So essentially I need to use the functions which check for the presence of messages without…
Steven Lu
  • 41,389
  • 58
  • 210
  • 364
1 2 3
99
100