3

I have dependency graph of operations and I use multiple queues to organize various streams of operations. E.g. peopleQueue, sitesQueue, sessionQueue

sessionQueue: loginOp, fetchUpdatedAccountOp peopleQueue: mostFrequentlyManagedClientsOp, remainingClientsOp sitesQueue: mostFrequentlyAccessedSitesOp, remainingSitesOp

dependencies:

*all* -> loginOp
remainingClientsOp -> mostFrequentlyManagedClientsOp
remainingSitesOp -> mostFrequentlyAccessedSitesOp

The current setup works: after login completes, all the other operations kick off mostFrequently* is a subset fetch that allows for quick app response, a subsequent op fetches much more data (sometimes in pages) in the background.

Recently I thought I'd add an operation that depended on all the leaf operations. This latest operation would act as a sentinel to tell me when the graph traversal had completed (firing it would cause on NSNotification post or something). So:

sentinelOp -> remainingClientsOp, remainingSitesOp, fetchUpdatedAccountOp

What I discovered, however, is that even though all its dependencies completed, the sentinel operation never started/fired. The sentinel, at the time was queued, on the sessionQueue (no particular reason).

After playing around in the debugger, I discovered that I could only get it to fire if the sentinel depended on only operations that were on the same queue.

I finally got the sentinel to run by introducing a 4th queue for just that operation. The sentinel depends on the other 3 leaf operations in their respective queues and then gets called when they all complete.

I can go with this working model but it really bothers me. The Apple docs for both mac and iOS suggest that inter-queue dependency should work.

I will need to extend the graph a bit further so it troubles me that using an existing queue for inter-queue dependencies prevents the operation from executing. Clearly, inter-queue dependencies work to some extent because I got loginOp to be the root dependency for other operations regardless of their queues in the first place.

What am I doing wrong by placing the sentinel operation on one of the existing 3 queues?

Kakashi Sensei
  • 131
  • 8
  • 1
    Are your queues parallel or serial? Have you tried on other existing queues? It's possible that the queue was blocked creating a defacto dependency. – Brian Nickel Dec 13 '13 at 22:23
  • 1
    All three queues are parallel. Not sure I get your question about existing queues - do you mean put the operations on something like [NSOperationQueue mainQueue]? Currently all queues are just created using [[NSOperationQueue alloc] init]; – Kakashi Sensei Dec 14 '13 at 01:08

1 Answers1

2

I resolved this issue by using only 1 queue. I still can't understand what was wrong with the original implementation but I learned a couple of things that removed the need for multiple queues.

First, it's somewhat straightforward to observe the queues pending operations count using KVO. This is how I was able to do away with the sentinel (see Reference).

Second, I was maintaining several queues to logically separate out related operations. With one queue, I achieved pretty much the same results by composing the operations generation method into helper methods, 1 for each logical unit and then enqueuing all the operations returned by the helper.

I am not sure if there are performance implications to going from 3 queues to 1. As far as I can tell, so long as the operations are concurrent and the queue has no restrictions on current execution, it shouldn't matter whether the operations are distributed among multiple queues or all on the same queue.

Siempay
  • 876
  • 1
  • 11
  • 32
Kakashi Sensei
  • 131
  • 8