2

I have a camera session and i am taking images from the buffer:

   -(void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection
{
    CVPixelBufferRef pixelBuffer = (CVPixelBufferRef)CMSampleBufferGetImageBuffer(sampleBuffer);

    CIImage *ciImage = [CIImage imageWithCVPixelBuffer:pixelBuffer];        
    //rotate image 90°
    ciImage = [ciImage imageByApplyingTransform:CGAffineTransformMakeRotation(-M_PI/2.0)];
}

And i am applying a filter on the image, and i want it to apply the filter on another queue, it is not thread-safe so if i take images to fast it will blend the images together (50/50 from left to right, i think), but i am trying to make it thread-safe, and it will not work by using NSLock or NSRecursiveLock because it will blend the images together.

dispatch_async(filterQueue, ^{
            CIImage *scaleImage = [CIFilter filterWithName:@"CILanczosScaleTransform" keysAndValues:kCIInputImageKey, ciImage, @"inputScale", [NSNumber numberWithFloat:0.5], nil].outputImage;

            CGImageRef cgImage = [imageContext createCGImage:scaleImage fromRect:scaleImage.extent];
            [self.pictureArray addObject:[UIImage imageWithCGImage:cgImage]];
            CGImageRelease(cgImage);
        });

Can someone help me? i have not much knowledge in how to make code a thread-safe

Images blend like this: https://i.stack.imgur.com/wjrIl.png

Jóhann Forberg
  • 61
  • 1
  • 9

2 Answers2

2

First, the only part of the above code that isn't thread safe is the call to addObject:. You can make your code thread-safe by moving your addObject: call to the main thread, or by making pictureArray accesses more thread-safe. Let's look at both.

Move the call

This is almost certainly how you'd want to do it.

dispatch_async(filterQueue, ^{
  CIImage *scaleImage = [CIFilter filterWithName:@"CILanczosScaleTransform" keysAndValues:kCIInputImageKey, ciImage, @"inputScale", [NSNumber numberWithFloat:0.5], nil].outputImage;

  CGImageRef cgImage = [imageContext createCGImage:scaleImage fromRect:scaleImage.extent];
  UIImage *image = [UIImage imageWithCGImage:cgImage];
  CGImageRelease(cgImage);
  cgImage = NULL; // Always set to NULL after you release something
  dispatch_async(dispatch_get_main_queue(), ^{
    [self.pictureArray addObject:image];
  });
});

Note that I'm doing as much as I can on the background thread. I just move the final addObject: to the main thread.

Thread-safe addImage:

If you call addObject: often from background threads, it may be nice to hoist this out into its own method like this:

- (void)addImageOnMainThread:(UIImage *)image {
  dispatch_async(dispatch_get_main_queue(), ^{
    [self.pictureArray addObject:image];
  });
}

Thread-safe pictureArray:

This is better if mutation is very common and locking is a performance problem. Using barriers leads to very fast accesses, at the expense of more complicated code.

- (UIImage *)imageAtIndex:(NSUInteger)index {
  UIImage *result = nil;
  dispatch_sync(self.pictureArrayQueue, 
    ^{ result = self.pictureArray[index]; });
  return result;
}

- (void)addImage:(UIImage *)image {
  dispatch_barrier_async(self.pictureArrayQueue, 
    ^{ [self.pictureArray addObject:image]; });
}

Note that there's a new dispatch queue here for accessing pictureArray. You can build more readers and writers (like removeImageAtIndex:, etc.) All readers use dispatch_sync. All writers use dispatch_barrier_async. This requires more code, and you must never directly access pictureArray except through these methods. The advantage is that it's much faster if mutation is very common.

Rob Napier
  • 286,113
  • 34
  • 456
  • 610
  • I think the ciImage is not thread-safe because i think ciImage get changed while the filter is creating scaleImage. It must be, because the result is 2 image blended together.. – Jóhann Forberg Dec 10 '12 at 16:16
  • And i have tried to set the addObject: code on main thread, and the result are the same, 2 image blended together.. :( – Jóhann Forberg Dec 10 '12 at 16:22
  • Where does "imageContext" come from? Are you accessing it on multiple threads? – Rob Napier Dec 10 '12 at 16:35
  • I am accessing imageContext from multiple threads, but I change it to: `@property (atomic, strong) CIContext *imageContext;` **and** `viewDidLoad - {self.imageContext = [CIContext contextWithOptions:nil];}` **and** `CGImageRef cgImage = [self.imageContext createCGImage:scaleImage fromRect:scaleImage.extent];` But it still get the same result.. – Jóhann Forberg Dec 10 '12 at 17:19
  • Try bracketing you call to imageWithCVPixelBuffer: with CVPixelBufferLockBaseAddress and CVPixelBufferUnlockBaseAddress. I haven't seen documentation that says that's necessary here, but it's the kind of problem I'd expect if you left it out when it was needed. – Rob Napier Dec 10 '12 at 19:10
0

Use @synchronized(self) to make a code fragment thread safe.

For example:

-(void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)
   //The following code will be thread safe.
   @synchronized(self) {
      CVPixelBufferRef pixelBuffer = (CVPixelBufferRef)CMSampleBufferGetImageBuffer(sampleBuffer);

      CIImage *ciImage = [CIImage imageWithCVPixelBuffer:pixelBuffer];        
      //rotate image 90°
       ciImage = [ciImage imageByApplyingTransform:CGAffineTransformMakeRotation(-M_PI/2.0)];
    }
}

The @synchronized statement will lock this section of code to be used for a single thread at once. You can apply it anywhere where you need code to be single threaded.

Hope that helps.

Boris Prohaska
  • 912
  • 6
  • 16