Persistent Blocking Queue in Java?

Question

TL;DR; I need to know if there's a lib with persistent blocking queue that performatic.

I hava a classic producer/consumers program. They share a LinkedBlockingQueue to share the data, and I use BlockingQueue#take method in the Consumers, as I need them to live forever waiting for new elements.

The problem is I have LOTS of data and I can't lose them. Even after the consumers stops, the producer can persist to generate some data. I am thinking about implementing my BlockingQueue ta uses H2 behind to store/get the data after some threshold is reached. My main problem is that I need performance and I need to consume the elements in the order they are created.

Is there an implementation of persistent blocking queue that I can use for something like this? If it doesn't, any sugestions for me to achieve something like this?

score 3 · Answer 1 · answered Mar 07 '13 at 22:29

3

You can try ActiveMQ. The ActiveMQ can write to your file system so if the producer is generating many more elements than the consumer can take you either have a lot of blocking (on whatever the upper bound of the queue is) or excessive data (if there is no upper bound to the queue).

answered Mar 07 '13 at 22:29

John Vint

39,695
7
78
108

but does it guarantee order? – Caesar Ralf Mar 07 '13 at 22:36

score 3 · Accepted Answer · answered Mar 08 '13 at 00:23

3

I would use ActiveMQ lib and Spring JMS, here is a usage example

start broker

BrokerService broker = new BrokerService();
broker.addConnector("tcp://localhost:61616");
broker.start();

read msg

ConnectionFactory cf = new ActiveMQConnectionFactory("tcp://localhost:61616");
JmsTemplate t = new JmsTemplate(cf);
Message msg = t.receive();

send message

ConnectionFactory cf = new ActiveMQConnectionFactory("tcp://localhost:61616");
JmsTemplate t = new JmsTemplate(cf);
t.send("test", new MessageCreator() {
  public Message createMessage(Session session) throws JMSException {
    return session.createTextMessage("test");
  }
});

answered Mar 08 '13 at 00:23

Evgeniy Dorofeev

133,369
30
199
275

I can't use spring, because it's an old software :/ Anyway, does JMS persists the Queue? Didn't know that. Don't I need to register a topic? – Caesar Ralf Mar 11 '13 at 13:33
1) Spring is just for convinience, you can work directly with JMS. 2) It's not JMS but Messaging Provider like ActiveMQ who persists messages, the message sender decides whether a message should be persisted or not when it sends it, there is a parameter - DeliveryMode – Evgeniy Dorofeev Mar 11 '13 at 14:10

score 1 · Answer 3 · answered Mar 07 '13 at 23:05

1

Have you come across Amazon SQS its an unbounded queue and very fast, It guarantees order.How long do you want to persist the data for?

answered Mar 07 '13 at 23:05

dineshr

329
2
6

I need it for an intranet, but thanks for the answer :D Didn't know SQS – Caesar Ralf Mar 11 '13 at 13:32

score 0 · Answer 4 · answered Dec 04 '18 at 22:25

0

You can use any JMS implementation for supporting excess incoming data. This is a producer consume problem and jms is designed for this.

answered Dec 04 '18 at 22:25

Akash5288

1,919
22
16

Persistent Blocking Queue in Java?

4 Answers4