Spring / RabbitMQ : transaction management

Question

To simplify my problem, I have

App1 with @Transactionnal method createUser():

Insert new user in database
Add async message in RabbitMQ so that the user receives a notification mail
(potentially some additional code, but not much)

App2 with RabbitMQ message consumer

Consumes messages on the mailing queue in real-time
Read mail data in database
Send mail

The problem is that sometimes, App2 tries to consume the RabbitMQ message before the transaction is even committed on App1. This means that App2 can't read the mail data on database because the user is not yet created.

Some solutions may be:

Use READ_UNCOMMITED isolation level on App2
Add some delay in RabbitMQ messages delivery (or some RetryTemplate on the consumer)
Change the way we send emails...

I've seen there is a RabbitTransactionManager in Spring, but I can't understand how it is supposed to work. Internals of transaction handling stuff has always seemed to be a bit hard to understand and the documentation doesn't help so much either.

Is there a way to do something like this?

Add a message to a RabbitMQ queue in a @Transactionnal method
When the transaction ends, the message is committed to the queue, and the changes are committed to the database
So that the message can't be consumed before the DB transaction ends

How? And what to expect for example if I send synchronous RabbitMQ messages instead of asynchronous messages? Would it block the thread waiting for a response or something? Because we do send sync and async messages for different usecases.

am i right that both apps consume messages from the same queue? — pinepain, Jul 02 '13 at 18:05
No, App1 put messages in a queue and App2 consumes messages in that queue — Sebastien Lorber, Jul 03 '13 at 09:26
Did you manage to solve this problem somehow? I am facing the same issue. — Piotr, Dec 31 '13 at 17:42

score 3 · Answer 1 · answered Feb 05 '16 at 16:30

I know this is late but I had this same problem due to my limited understanding of @Transactional at the time. So this is more for anyone else you happens to stumble upon this.

When use @Transactional to save data to the database the save to the database doesnt actually happen until the method returns, and not when the save is called.

So if you have a method like

@Transactional(readOnly=false)
public void save(Object object) { //Object should be one of your entities
  entityManager.persist(object); //or however you have it set up
  rabbitTemplate.convertAndSend(message); //again - however yours is
}

even tho you call the persist on the object before you put the message on the queue, the persist will not actually happen until the method returns, thus causing the message to be put on the queue before the method returns and before the data is actually in the database.

It is possible to nest @Transactional methods (tho its not straight forward) can put the message on the queue after the save() method returns. However you cannot put the message on the queue and expect it to not be consumed. Once its there is gone. So delay putting it on the queue if you need to.

If you want to receive a response from the queue in a sync manner. In my function example - you can do this, however it will only make it take longer to actually persist the data, as it will be waiting for the response from the worker before the method can return and actually persist the data. (also remember that receiving a response from a queued message has a timeout).

So my advise is to not put those 2 operations in the same @Transactional

score 1 · Answer 2 · answered Jul 03 '13 at 10:31

I'm not really familiar with @Transactionnal and spring, but in AMQP standard message queuing is not transactional operation, so you have to store data in db (if db connection is transactional - commit transaction) and only after that send message to broker.

The correct workflow looks for me like App1: createUser -> notifyUser; App2: listenForNotifications

Spring / RabbitMQ : transaction management

2 Answers2