Concurrency safe pivoting of Azure Service Fabric Reliable Collections

Question

I have a service that has a bunch of incoming Orders that I store into an IReliableDictionary. As new orders come in, I need to update and maintain a separate sorted list of those orders throughout the lifetime of my program.

How do I efficiently and concurrency pivot the data in the IReliableDictionary into a different sorted collection such that I avoid deadlocks and don't have to rebuild the sorted list from scratch every time a new order comes in?

EDIT: After looking at the documentation I believe I can achieve this by updating my local memory sorted collection after an update before a commit.

        using (var tx = this.StateManager.CreateTransaction())
        {
            bool addOk = await orderDictionary.TryAddAsync(tx, 123, someOrderToAdd);
            if (addOk)
            {
                this.SortedOrders.Add(someOrderToAdd);
            }
            await tx.CommitAsync();
        }

Can someone confirm that my understanding of the documentation is correct, and that something like the above implementation will not cause concurrency issues?

score 1 · Accepted Answer · answered May 30 '18 at 09:55

1

You can use reliable collections notifications and store in sorted list. This will ensure that you never miss out on events related to dictionary.

answered May 30 '18 at 09:55

teeboy

408
3
13

What's the recommended usage of that? Is the notification in-order/concurrency safe? Or, should I lock on the sorted list and then insert (assuming the reliabledictionary doesn't maintain some sort of Update lock during the notification)? – rysama May 31 '18 at 23:34
If u use a concurrentdictionary, you don't need a lock. You do need a lock for sorted list. The events arrive in order. Don't do expensive operations in the event handler. – teeboy Jun 02 '18 at 02:51

score 0 · Answer 2 · answered May 28 '18 at 06:49

If it's only for the lifetime of the process, and if it's no problem to have only Orders from within the partition, maybe you can use a regular concurrent collection. For example, a ConcurrentQueue<Order>.

You can also use an event driven approach, signalling a timestamped OrderReceivedEvent every time an Order is received. Then you can subscribe to those events in a separate service that stores them somewhere.

Or you can use a distributed cache like Redis (in a Container) to keep Orders temporarily.

Another option could be to use SQL server for Order data, instead of reliable collections.

Concurrency safe pivoting of Azure Service Fabric Reliable Collections

2 Answers2