-1

What I want to do is create an indirect queue targeting the main queue.

dispatch_queue_t myQueue = dispatch_queue_create("com.mydomain.my-main-queue", NULL);
dispatch_set_target_queue(myQueue, dispatch_get_main_queue());

My ultimate goal is to use the queue as the underlyingQueue property of an NSOperationQueue, because Apple's documentation clearly states not to use dispatch_get_main_queue(). Though using an indirect queue it technically is following the documentation.

The reason for all this is because NSOperationQueue.mainQueue is not a safe for asynchronous operations, because it is globally accessible and it's maxConcurrentOperationCount is set to 1. So can easily shoot yourself in the foot with this operation queue.

Update 1

It seems there is a lot of confusion about the basis of what this question assumes an "asynchronous NSOperation" is. To be clear this is based on the concepts in this WWDC session The particular concept is using "operation readiness" and dependency management to manage the tasks in your app, which means asynchronous NSOperations are added to NSOperationQueues to take advantage of this. If you take these concepts to the spirit of this question hopefully the reasoning will make more sense, and you can focus on comparing and contrasting the solution with other ones.

Update 2 - Example of issue:

// VendorManager represents any class that you are not in direct control over.

@interface VendorManager : NSObject
@end

@implementation VendorManager

+ (void)doAnsyncVendorRoutine:(void (^)(void))completion {
    // Need to do some expensive work, make sure we are off the main thread
    dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND 0), ^(void) {
        // Some off main thread background work
        sleep(10);
        // We are done, go back to main thread
        [NSOperationQueue.mainQueue addOperationWithBlock:completion];
    });
}

@end


// MYAsyncBoilerPlateOperation represents all the boilerplate needed
// to implement a useful asnychronous NSOperation implementation.

@interface MYAlertOperation : MYAsyncBoilerPlateOperation
@end

@implementation MYAlertOperation

- (void)main {

    UIAlertController *alertController = [UIAlertController alertControllerWithTitle:"Vendor"
                                                                             message:"Should vendor do work?"
                                                                      preferredStyle:UIAlertControllerStyleAlert];
    __weak __typeof(self) weakSelf = self;
    [alertController addAction:[UIAlertAction actionWithTitle:@"Yes"
                                                        style:UIAlertActionStyleDefault
                                                      handler:^(UIAlertAction *action) {
                                                          [VendorManager doAnsyncVendorRoutine:^{
                                                              // implemented in MYAsyncBoilerPlateOperation
                                                              [weakSelf completeThisOperation];
                                                          }];
                                                      }]];
    [alertController addAction:[UIAlertAction actionWithTitle:@"No"
                                                        style:UIAlertActionStyleDefault
                                                      handler:^(UIAlertAction *action) {
                                                          [weakSelf cancel];
                                                      }]];

    [MYAlertManager sharedInstance] presentAlert:alertController animated:YES];
}

@end

// MYAlertOperation will never complete.
// Because of an indirect dependency on operations being run on mainQueue.
// This example is an issue because mainQueue maxConcurrentOperationCount is 1.
// This example would not be an issue if maxConcurrentOperationCount was > 1.

[NSOperationQueue.mainQueue addOperation:[[MYAlertOperation alloc] init]];

Update 3 - Example 2:

I am not showing the implementation of MyAsyncBlockOperation but you can use this as what it's based on in Swift.

// operation.asynchronous is YES, and things are implemented correctly for state changes.
MyAsyncBlockOperation *operation = [MyAsyncBlockOperation new];
__weak MyAsyncBlockOperation *weakOperation = operation;
// executionBlock is simply invoked in main
// - (void)main { self.executionBlock() };
operation.executionBlock = ^{
    UIAlertController *alertController = [UIAlertController alertControllerWithTitle:@"Vendor"
                                                                             message:@"Should vendor do work?"
                                                                      preferredStyle:UIAlertControllerStyleAlert];

    [alertController addAction:[UIAlertAction actionWithTitle:@"Yes"
                                                        style:UIAlertActionStyleDefault
                                                      handler:^(UIAlertAction *action) {
                                                          [[NSOperationQueue mainQueue] addOperationWithBlock:^{
                                                              NSLog(@"Never called");
                                                              [weakOperation completeWithSuccess];
                                                          }];
                                                      }]];

    [alertController addAction:[UIAlertAction actionWithTitle:@"No"
                                                        style:UIAlertActionStyleDefault
                                                      handler:^(UIAlertAction *action) {
                                                          [weakOperation cancel];
                                                      }]];

    [[UIApplication sharedApplication].keyWindow.rootViewController presentViewController:alertController animated:YES completion:nil];
}];

operation.completionBlock = ^{
    NSLog(@"If YES, Never called. If NO, called.");
};

[[NSOperationQueue mainQueue] addOperation:operation];

So I thought, why not have another NSOperationQueue? One with it's underlyingQueue set to the previously mentioned indirect GCD queue (still following the documentation). So we can have a concurrent NSOperationQueue, legally targeting the serial main GCD queue, and ultimately ensuring the operations run on the main thread.

Let me know if you want clarification, here is an example of the full code:

NSOperationQueue *asyncSafeMainQueue = [[NSOperationQueue alloc] init];
asyncSafeMainQueue.qualityOfService = NSQualityOfServiceDefault; // not needed, just for clarity
dispatch_queue_t underlyingQueue = dispatch_queue_create("com.mydomain.main-thread", NULL);
dispatch_set_target_queue(underlyingQueue, dispatch_get_main_queue());
asyncSafeMainQueue.underlyingQueue = underlyingQueue;

Now... there is a safe operation queue for asynchronous operations that need to run on the main thread, and without any unnecessary context switching.

Is it safe?

drkibitz
  • 507
  • 5
  • 15
  • "Now... there is a safe operation queue for asynchronous operations that need to run on the main thread, and without any unnecessary context switching." ... I'd suggest you describe the key aspects of these tasks you're running on the operation queue. This feels like an XY-problem, where you're asking us to comment on your proposed solution to some other problem. We can't tell you how to solve that original challenge without understanding what you're doing there. – Rob Jan 03 '18 at 20:37
  • The tasks are anything that require the main thread, which is anything that touches UIApplication or other UIKit objects. Or in the third party sense, Chromecast SDK objects. Using NSOperations and NSOperationQueues for tasks that must be run on the main thread doesn't seem like a horrible idea. It's just difficult to do with these objects without a bunch of context switching. – drkibitz Jan 03 '18 at 20:41
  • Think of it like an transitionOperationQueue, and one operation is a single transition. When the transition completes the operaition completes. Then other transition operations can depend on other transition operations. Becasuse these are async operations, they are unsafe to depend on each other on the mainQueue. But they all need to be run on the main thread. The the ultimate goal is to remove context switch from the usage of NSOperation(Queue) when using them for tasks needed to be run on the main thread. Does this help? – drkibitz Jan 03 '18 at 20:43
  • “Because these are async operations, they are unsafe to depend on each other on the mainQueue.” If, by “depend on each other”, you mean using `-[NSOperation addDependency:]`, then it's safe for operations on the main queue to depend on each other. – rob mayoff Jan 03 '18 at 20:50
  • 1
    @drkibitz - If you're performing tasks that are (a) themselves asynchronous (i.e. you find yourself having to wrap it in an asynchronous `NSOperation` subclass); and (b) require the main thread, then just use the main queue. I'd suggest you edit your question to supply a single, specific example of where you're planning on using operation queue for something that requires the main thread. Discussing this in the abstract is not likely to be fruitful. – Rob Jan 03 '18 at 20:54
  • "Present view operation", runs "dismiss other view operation". Both operations are are asynchronous, and added to mainQueue . How do you not see the issue there? – drkibitz Jan 03 '18 at 21:01
  • Ok I see, sorry I didn't mean "addDependency:". – drkibitz Jan 03 '18 at 21:02
  • I've updated with a concrete issue example – drkibitz Jan 03 '18 at 21:12
  • So, to clarify, `presentAlertWithOkAction:` blocks until `completeThisOperation` runs? If so, then yes, there's a deadlock, and no amount of `underlyingQueue` and `dispatch_set_target_queue` shenanigans will fix that. You either have to change `presentAlertWithOkAction:` to not block, or change `someAnsyncRoutine:` to not dispatch back to the main queue. – rob mayoff Jan 03 '18 at 21:45
  • That's deadlocking because you are calling `completeThisOperation` for one operation from within a separate operation on the same queue that obviously can't start until the first one finishes. – Rob Jan 03 '18 at 21:46
  • What do you mean by “blocks”? Also, if you pay attention to the names of the pseudo classes, I did not have control of what happened, the way you insinuate. This is common in third party code – drkibitz Jan 03 '18 at 22:02
  • My solution definitely fixes this issue. If you can't see it, I don't think my example was clear? No method being invoked there "blocks" the current execution context. But the mainQueue does have the max count set to 1, so it is an NSOperationQueue deadlock in that sense. It is a serial NSOperationQueue, with an underlying queue that is also serial. So serial twice over. My solution is to make a concurrent operation queue, with an underlyingQueue that is serial. It removes the issue with maxConcurrentOperationCount, but the operations themselves still execute serially on the main thread. – drkibitz Jan 03 '18 at 22:09
  • Update the example. Hopefully it is more clear. – drkibitz Jan 03 '18 at 22:42
  • @Rob If you can, please show me code that is "correct"? Because seriously, you've taken the spirit of my question to a place where you are judging my pseudo code for the reasoning of my question. A solution that is "convoluted" is relative to one's understanding of the problem. And If you are taking my example as concrete I don't know how else to explain the problem without a diagram and pictures, which at this point is superfluous. Long story short, the spirit of the question was maybe mainQueue should not have it's max count set to 1 anyway – drkibitz Jan 03 '18 at 22:59
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/162464/discussion-between-rob-and-drkibitz). – Rob Jan 03 '18 at 23:18
  • Based on reactions to this question, I think I will followup with another question. – drkibitz Jan 04 '18 at 01:07
  • @Rob I found your answer to this question. https://stackoverflow.com/questions/24456407/creating-a-method-to-perform-animations-and-wait-for-completion-using-a-semaphor Consider this. Would if instead of adding your AnimationOperation to animationQueue, it was instead added to [NSOperationQueue mainQueue]. This an the issue, you instead use animationQueue to not encounter it. But really this extra step would be unnecessary if mainQueue.maxConcurrentOperationCount was not set to 1. So my question is about removing a step in another way (since we can't touch the mainQueue). – drkibitz Jan 04 '18 at 08:39
  • The 2 steps I'm talking about is - 1 create an operationQueue that is not mainQueue, and 2, dispatch back to the mainQueue because of the nature of these kinds of operations (UI related operations). – drkibitz Jan 04 '18 at 08:43
  • @drkibitz - I've added your comments here to our [existing chat](http://chat.stackoverflow.com/rooms/162464/discussion-between-rob-and-drkibitz) as this is not the right place for prolonged kibitzing (sorry, I couldn't stop myself). – Rob Jan 04 '18 at 16:57

2 Answers2

0

I don't understand why you think mainQueue is not safe for asynchronous operations. The reasons you gave would make it unsafe for synchronous operations (because you could deadlock).

Anyway, I think it's a bad idea to try the workaround you're suggesting. Apple didn't explain (on the pages you linked) why you shouldn't set underlyingQueue to the main queue. I recommend you play it safe and follow the spirit of the prohibition rather than the letter.

Update

Looking now at your updated question, with example code, I see nothing that can block the main thread/queue, so there is no possibility of deadlocking. It doesn’t matter that mainQueue has a macConcurrentOperationCount of 1. I don’t see anything in your example that requires or benefits from the creation of a separate NSOperationQueue.

Also, if the underlyingQueue is a serial queue (or has a serial queue anywhere in its target chain), then it doesn’t matter what you set maxConcurrentOperationCount to. The operations will still run serially. Try it yourself:

@implementation AppDelegate {
    dispatch_queue_t concurrentQueue;
    dispatch_queue_t serialQueue;
    NSOperationQueue *operationQueue;
}

- (void)applicationDidFinishLaunching:(NSNotification *)aNotification {
    concurrentQueue = dispatch_queue_create("q", DISPATCH_QUEUE_CONCURRENT);
    serialQueue = dispatch_queue_create("q2", nil);
    operationQueue = [[NSOperationQueue alloc] init];

    // concurrent queue targeting serial queue
    //dispatch_set_target_queue(concurrentQueue, serialQueue);
    //operationQueue.underlyingQueue = concurrentQueue;

    // serial queue targeting concurrent queue
    dispatch_set_target_queue(serialQueue, concurrentQueue);
    operationQueue.underlyingQueue = serialQueue;

    operationQueue.maxConcurrentOperationCount = 100;

    for (int i = 0; i < 100; ++i) {
        NSOperation *operation = [NSBlockOperation blockOperationWithBlock:^{
            NSLog(@"operation %d starting", i);
            sleep(3);
            NSLog(@"operation %d ending", i);
        }];
        [operationQueue addOperation:operation];
    }
}

@end
rob mayoff
  • 375,296
  • 67
  • 796
  • 848
  • There is a difference between synchronous and serial, and asynchronous and concurrent. NSOperation.mainQueue is global and serial. Your async operation can't always be sure that an asynchronous routine it depends on for completion doesn't eventually put another operation on the mainQueue. If this is the case, it's not a deadlock in a classic sense, but your async operation will never complete if it is indirectly waiting on the routine's operation that it added, whether synchronous or asynchronous, because the mainQueue is serial. – drkibitz Jan 03 '18 at 20:50
  • I also find it curious that the documentation specifically mentions "the value returned by..." specific method, dispatch_get_main_queue. – drkibitz Jan 03 '18 at 20:53
  • 1
    What do you mean by “an asynchronous routine it depends on”? How does the first operation depend on the second? `dispatch_sync`? `dispatch_semaphore_wait`? `dispatch_group_wait`? `-[NSOperationQueue waitUntilAllOperationsAreFinished]`? `-[NSOperation waitUntilFinished]`? Please spell out a concrete example instead of speaking in abstractions. – rob mayoff Jan 03 '18 at 20:55
  • Please see concrete example added to question – drkibitz Jan 03 '18 at 21:17
  • You apparently are not familiar with asynchronous NSOperations and how they behave in an NSOperation queue. I don’t know how else to explain this without going in great detail. Please feel free to start a chat. maxConcurrentOperationCount still has an effect on things, and no there is no deadlock in classical sense, but there is a deadlock in NSOperations, in that one will never complete because the other will never start. – drkibitz Jan 04 '18 at 02:07
  • I have tested what I said: an `NSOperationQueue` whose `underlyingQueue` is a serial `dispatch_queue_t` (or has a serial `dispatch_queue_t` anywhere in its target chain) cannot execute multiple operations simultaneously. I have updated my answer with test code to demonstrate it. If you can post a counterexample, I'd love to see it. If you can post an example of “a deadlock in NSOperations”, I'd love to see that also. I do not think there is a deadlock of any kind in the code in your question. – rob mayoff Jan 04 '18 at 02:49
  • Also please take note of [How to create a Minimal, Complete, and Verifiable example](https://stackoverflow.com/help/mcve). In particular, your example code lacks completeness, so it's difficult to be sure I understand all the parts of your question/problem. – rob mayoff Jan 04 '18 at 02:53
  • It seems you are approaching the answer as if all operations are synchronous when in queues, and asynchronous when out of queues. Have you seen the video and example code here from WWDC a couple years ago? https://github.com/pluralsight/PSOperations The entire concept of "Advanced NSOperations" is using asynchronous operations to manage task dependencies which means you must add them to operation queues to take advantage of "readiness" concept. If you take this as the basis of what an "asynchronous operation" is, maybe all of this will make more sense to you? – drkibitz Jan 04 '18 at 04:10
  • I've updated my question to include those references – drkibitz Jan 04 '18 at 04:25
  • I don't think I am approaching the answer the way your first sentence indicates, but based on that sentence, I'm not sure we are using the same definitions of “synchronous” and “asynchronous”. At any rate, I don't think I have anything else to say on this subject without a complete, verifiable example of the problem you're trying to solve. Sorry I couldn't be more helpful. – rob mayoff Jan 04 '18 at 04:27
  • Thank you for your time. If we do have the same definition I would have hoped to continue it. If you do not, that's ok still appreciate it.I have added "Example 2" to my question. If you decide look at it, pay attention to the NSLog lines. I can't give a runnable example because I have proprietary code for much of my implementations, so I am showing things as close as I can without completely showing and using an implementation of an async operation. – drkibitz Jan 04 '18 at 04:45
0

Hmm.. This crashes very badly in Swift-4 if using setTarget instead of the designated constructor..

If you use Objective-C bridging, then you can do:

@interface MakeQueue : NSObject
+ (NSOperationQueue *)makeQueue:(bool)useSerial;
@end

@implementation MakeQueue
+ (NSOperationQueue *)makeQueue:(bool)useSerial {
    dispatch_queue_t serial = dispatch_queue_create("serial", nil);
    dispatch_queue_t concurrent = dispatch_queue_create("concurrent", DISPATCH_QUEUE_CONCURRENT);

    dispatch_queue_t queue = useSerial ? serial : concurrent;
    dispatch_set_target_queue(queue, dispatch_get_main_queue());

    NSOperationQueue *opq = [[NSOperationQueue alloc] init];
    opq.underlyingQueue = queue;
    opq.maxConcurrentOperationCount = 8;
    return opq;
}
@end

and if using Swift, you have:

func makeQueue(_ useSerial: Bool) -> OperationQueue? {

    let testCrash: Bool = false
    var queue: DispatchQueue!

    if testCrash {
        let serial = DispatchQueue(label: "serial")
        let concurrent = DispatchQueue(label: "concurrent", attributes: .concurrent)
        queue = useSerial ? serial : concurrent
        queue.setTarget(queue: DispatchQueue.main)
    }
    else {
        let serial = DispatchQueue(label: "serial", qos: .default, attributes: .init(rawValue: 0), autoreleaseFrequency: .inherit, target: DispatchQueue.main)
        let concurrent = DispatchQueue(label: "concurrent", qos: .default, attributes: .concurrent, autoreleaseFrequency: .inherit, target: DispatchQueue.main)
        queue = useSerial ? serial : concurrent
    }

    let opq = OperationQueue()
    opq.underlyingQueue = queue
    opq.maxConcurrentOperationCount = 8;
    return opq
}

So now we test it:

class ViewController : UIViewController {

    override func viewDidLoad() {
        super.viewDidLoad()

        //Test Objective-C
        let operationQueue = MakeQueue.makeQueue(false)!
        operationQueue.addOperation {
            self.download(index: 1, time: 3)
        }

        operationQueue.addOperation {
            self.download(index: 2, time: 1)
        }

        operationQueue.addOperation {
            self.download(index: 3, time: 2)
        }


        //Test Swift
        let sOperationQueue = makeQueue(false)!
        sOperationQueue.addOperation {
            self.download(index: 1, time: 3)
        }

        sOperationQueue.addOperation {
            self.download(index: 2, time: 1)
        }

        sOperationQueue.addOperation {
            self.download(index: 3, time: 2)
        }
    }

    func download(index : Int, time: Int){
        sleep(UInt32(time))
        print("Index: \(index)")
    }
}

In any case, it doesn't seem to matter what the maxConcurrentOperations are.. if the underlying queue is serial, then setting this value seems to do NOTHING.. However, if the underlying queue is concurrent, it places a limit on how many operations can be ran at once.

So all in all, once the underlying queue is MainQueue or any serial-queue, all the operations get submitted to it (serially) and they block (it waits because it is serial queue).

I'm not sure what the point of the underlying queue is if we're already using a designated queue anyway.. but in any case, setting it to main causes everything to run on the main queue and serially regardless of max concurrent count.

This: https://gist.github.com/jspahrsummers/dbd861d425d783bd2e5a is the only use-case that I could find.. AND that you can independently resume/suspend tasks on your custom queue even if its underlying queue is main or some other queue. AND suspending/resuming the one queue that all other queues target, will in turn suspend/resume all other queues.

Brandon
  • 22,723
  • 11
  • 93
  • 186
  • Thanks Brandon for jumping in the discussion. I just mentioned on a previous comment that the spirit of an "asynchronous operation" in this question is based on the concepts here https://github.com/pluralsight/PSOperations and the corresponding WWDC session. The entire concept of "Advanced NSOperations" is using asynchronous operations to manage task dependencies which means you must add them to operation queues to take advantage of the "readiness" concept. If you take this as the basis of what an "asynchronous operation" is here, maybe all of this will make more sense. – drkibitz Jan 04 '18 at 04:16
  • I've updated my question to include those references – drkibitz Jan 04 '18 at 04:25