AVAssetImageGenerator performance issue

Question

I am trying to create a video gallery.

To display videos I am using a UICollectionView. Each UICollectionViewCell has a background with video thumbnail. To generate a video thumbnail I am using a method with logic:

AVURLAsset *asset = [[AVURLAsset alloc] initWithURL:url options:nil];

AVAssetImageGenerator *generator = [[AVAssetImageGenerator alloc] initWithAsset:asset];

generator.appliesPreferredTrackTransform = YES;

CMTime time = CMTimeMakeWithSeconds(0,15);

AVAssetImageGeneratorCompletionHandler handler = ^(CMTime timeRequested, CGImageRef image, CMTime timeActual, AVAssetImageGeneratorResult result, NSError *error)
{
    NSLog(@"handler^()");

    if (result == AVAssetImageGeneratorSucceeded)
    {
        thumbnail = [UIImage imageWithCGImage: image];

        success(thumbnail);
    }

    else
    {
        failure(error);
    }
};

CGSize maximumSize = CGSizeMake(CLIPBOARD_COLLECTION_VIEW_CELL_WIDTH, CLIPBOARD_COLLECTION_VIEW_CELL_HEIGHT);

generator.maximumSize = maximumSize;

NSLog(@"generateCGImagesAsynchronouslyForTimes:");
[generator generateCGImagesAsynchronouslyForTimes:[NSArray arrayWithObject:[NSValue valueWithCMTime:time]] completionHandler:handler];

I have noticed that generateCGImagesAsynchronouslyForTimes doesn't work fully asynchronously. There a time space between this method calls. This causes a big lag while I am loading table view cells. If I comment line [generator generateCGImagesAsynchronouslyForTimes:[NSArray arrayWithObject:[NSValue valueWithCMTime:time]] completionHandler:handler] then there are no visible lag (nor images).

How can I solve this performance issue?

Can you specify what exactly lags? Does the scrolling performance suffer or does it just take a long time between the request and when the handler is called? Can you add the results of the Time Profiler instrument? — jszumski, Apr 29 '15 at 01:49
There are time spaces between `generateCGImagesAsynchronouslyForTimes` calls... — Daumantas Versockas, Apr 29 '15 at 11:20
@Sam, as far as I figured out, asynchronous image generation doesn't work as I expect. Generator process only limited number of images at once. It might be because it is to "heavy" process for iOS to handle in parallel. My temporary (until I figure out better one) solution is to put tasks in queues and call generator with limited number of images at once. — Daumantas Versockas, May 24 '16 at 05:07
@TomKortney Thanks for your answer. Could you please post your temporary solution? I have not been able to find a solution.. — Sam, May 24 '16 at 13:20

score 3 · Answer 1 · answered Apr 28 '15 at 20:32

The AVAssetImageGeneratorCompletionHandler block is not guaranteed to be called on the main thread:

Concurrent Programming with AV Foundation

Callouts from AV Foundation—invocations of blocks, key-value observers, and notification handlers—are not guaranteed to be made on any particular thread or queue. Instead, AV Foundation invokes these handlers on threads or queues on which it performs its internal tasks. You are responsible for testing whether the thread or queue on which a handler is invoked is appropriate for the tasks you want to perform. If it’s not (for example, if you want to update the user interface and the callout is not on the main thread), you must redirect the execution of your tasks to a safe thread or queue that you recognize, or that you create for the purpose.

Does calling success and failure with dispatch_async fix the issue?

dispatch_async(dispatch_get_main_queue(), ^{
    if (success) {
        success();
    }
)};

`dispatch_async` adding doesn't solve this performance issue. I am still having lags, even with `dispatch_asinc`. — Daumantas Versockas, Apr 28 '15 at 20:52

score 0 · Answer 2 · answered Jun 24 '16 at 06:43

There are a couple of ways to redirect the asynchronous image generation request to another thread, one of them being the encapsulation of the function within an NSBlockOperation added to an NSOperationQueue with its underlying queue set to what is most appropriate for your app.

The most appropriate queue should be relatively easy to guess, if you don't know, as there are only a few choices there.

AVAssetImageGenerator performance issue

2 Answers2