TxSelect and TransactionScope

Question

Recently, I've been checking out RabbitMQ over C# as a way to implement pub/sub. I'm more used to working with NServiceBus. NServiceBus handles transactions by enlisting MSMQ in a TransactionScope. Other transaction aware operations can also enlist in the same TransactionScope (like MSSQL) so everything is truly atomic. Underneath, NSB brings in MSDTC to coordinate.

I see that in the C# client API for RabbitMQ there is a IModel.TxSelect() and IModel.TxCommit(). This works well to not send messages to the exchange before the commit. This covers the use case where there are multiple messages sent to the exchange that need to be atomic. However, is there a good way to synchronize a database call (say to MSSQL) with the RabbitMQ transaction?

@kzhen I'm not really worried about performance. Consistency is important though. I'll be using durable exchanges and queues. Throughput will not be that high, maybe 50-100,000 messages per day. — Davin Tryon, Aug 04 '12 at 09:01

score 16 · Answer 1 · answered Oct 30 '13 at 15:33

You can write a RabbitMQ Resource Manager to be used by MSDTC by implementing the IEnlistmentNotification interface. The implementation provides two phase commit notification callbacks for the transaction manager upon enlisting for participation. Please note that MSDTC comes with a heavy price and will degrade your overall performance drastically.

Example of RabbitMQ resource manager:

sealed class RabbitMqResourceManager : IEnlistmentNotification
{
    private readonly IModel _channel;

    public RabbitMqResourceManager(IModel channel, Transaction transaction)
    {
        _channel = channel;
        _channel.TxSelect();
        transaction.EnlistVolatile(this, EnlistmentOptions.None);
    }

    public RabbitMqResourceManager(IModel channel)
    {
        _channel = channel;
        _channel.TxSelect();
        if (Transaction.Current != null)
            Transaction.Current.EnlistVolatile(this, EnlistmentOptions.None);
    }

    public void Commit(Enlistment enlistment)
    {
        _channel.TxCommit();
        enlistment.Done();
    }

    public void InDoubt(Enlistment enlistment)
    {           
        Rollback(enlistment);
    }

    public void Prepare(PreparingEnlistment preparingEnlistment)
    {
        preparingEnlistment.Prepared();
    }

    public void Rollback(Enlistment enlistment)
    {
        _channel.TxRollback();
        enlistment.Done();
    }
}

Example using resource manager

using(TransactionScope trx= new TransactionScope())
{
    var basicProperties = _channel.CreateBasicProperties();
    basicProperties.DeliveryMode = 2;

    new RabbitMqResourceManager(_channel, trx);
    _channel.BasicPublish(someExchange, someQueueName, basicProperties, someData);
    trx.Complete();
}

interesting... But, in the end we went with a no MSDTC solution. Thanks for adding this to the question, though, hopefully others will find it useful. :) — Davin Tryon, Oct 30 '13 at 15:40
I use the same approach in my application. However, I found that Commit/Prepare methods of IEnlistmentNotification are called in a different thread (it seems to be a normal behavior in System.Transaction), that cause some problems with RabbitMQ because IModel should not be used across thread (from the official documentation). Did you experiment this problem? — Normand Bedard, Apr 10 '15 at 12:52
No, i didnt. I think they are referring to thread concurrency since the channel is not thread safe. Thus if more than one thread uses the channel at the same time you will get issues. — , Apr 11 '15 at 20:12

score 5 · Accepted Answer · answered Aug 04 '12 at 18:47

As far as I'm aware there is no way of coordinating the TxSelect/TxCommit with the TransactionScope.

Currently the approach that I'm taking is using durable queues with persistent messages to ensure they survive RabbitMQ restarts. Then when consuming from the queues I read a message off do some processing and then insert a record into the database, once all this is done I ACK(nowledge) the message and it is removed from the queue. The potential problem with this approach is that the message could end up being processed twice (if for example the message is committed to the DB but say the connection to RabbitMQ is disconnected before the message can be ack'd), but for the system that we're building we're concerned about throughput. (I believe this is called the "at-least-once" approach).

The RabbitMQ site does say that there is a significant performance hit using the TxSelect and TxCommit so I would recommend benchmarking both approaches.

However way you do it, you will need to ensure that your consumer can cope with the message potentially being processed twice.

If you haven't found it yet take a look at the .Net user guide for RabbitMQ here, specifically section 3.5

Yes, I agree with everything you wrote. In my particular case, though, the consumer is not the problem. I have a situation where the producer might commit database state and then publish a message. Again, however, I totally agree about the need for idempotent messages on the consumer side. Thanks for the heads up about TxSelect and TxCommit performance. — Davin Tryon, Aug 04 '12 at 21:55
Perhaps your approach could have the producer commit the db row then send a message then update the row to say the message has been sent when the server confirms receipt (http://www.rabbitmq.com/blog/2011/02/10/introducing-publisher-confirms/) then if your producer crashes when it comes back online it could look for rows that haven't had their messages published and then (re)send them — kzhen, Aug 06 '12 at 08:40
Yeah, I think you are right. We are going to plan to commit that the message needs to be sent with the original db transaction. Then have a dispatcher pick it up and send to Rabbit. Finally, we will commit that the message has been sent. Thanks for the link! — Davin Tryon, Aug 06 '12 at 09:05

score 0 · Answer 3 · answered May 28 '15 at 18:03

Lets say you've got a service bus implementation for your abstraction IServiceBus. We can pretend it's rabbitmq under the hood, but it certainly doesn't need to be.

When you call servicebus.Publish, you can check System.Transaction.Current to see if you're in a transaction. If you are and it's a transaction for a mssql server connection, instead of publishing to rabbit you can publish to a broker queue within sql server which will respect the commit/rollback with whatever database operation you're performing (you want to do some connection magic here to avoid the broker publish upgrading your txn to msdtc)

Now you need to create a service that needs to read the broker queue and do an actual publish to rabbit, this way, for very important things, you can gaurantee that your database operation completed previously and that the message gets published to rabbit at some point in the future (when the service relays it). its still possible for failures here if when committing the broker receive an exception occurs, but the window for problems is drastically reduced and worse case scenario you would end up publishing multiple times, you would never lose a message. This is very unlikely, the sql server going offline after receive but before commit would be an example of when you would end up at minimum double publishing (when the server comes on-line you'd publish again) You can build your service smart to mitigate some, but unless you use msdtc and all that comes with it (yikes) or build your own msdtc (yikes yikes) you are going to have potential failures, it's all about making the window small and unlikely to occur.

TxSelect and TransactionScope

3 Answers3

Linked