11

My Unity app uses the Camera feed for multiple systems that the app uses; such as Vuforia, ARKit, and just general Camera feed input.

The issue I'm having is that each one of these requests a different Pixel Format, which seems to cause an issue for the ARKit. This requires YUV, and I don't know how to natively switch the camera pixel format back to this. So I get this error when I try and launch ARKit after the others;

2017-08-21 08:51:38.749452+0100 ar_app[399:105849] [Sensor] Unsupported pixel format: 875704438

2017-08-21 08:51:38.749876+0100 ar_app[399:105849] [Session] Session did fail with error: Error Domain=com.apple.arkit.error Code=104 "Unsupported capture session configuration."

UserInfo={ NSLocalizedRecoverySuggestion=Make sure that the correct device and format are being used for capture.,

NSLocalizedDescription=Unsupported capture session configuration.,

NSLocalizedFailureReason=Input device and/or format of the provided capture session are not supported for the given configuration. }

At least, thats what I think the issue is; that it's not getting the format in YUV. Any help would be much appreciated. . Thanks.

Oliver Jones
  • 1,420
  • 7
  • 27
  • 43

2 Answers2

3

YUV is a color space, there are multiple formats.

ar_app[399:105849] [Sensor] Unsupported pixel format: 875704438

875704438 translates to 420v (NV12) or kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange: 

kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange = '420v'

So you actually have a YUV pixel format, specifically the biplanar with video range.

The ARKit docs do mention the biplanar YUV format but not the type of range:

The pixel buffer’s contents are encoded in a biplanar YCbCr (also called YUV) data format

However the Unity ARKit Plugin does a check for kCVPixelFormatType_420YpCbCr8BiPlanarFullRange in didUpdateFrame:

 if (CVPixelBufferGetPlaneCount(pixelBuffer) < 2 || CVPixelBufferGetPixelFormatType(pixelBuffer) != kCVPixelFormatType_420YpCbCr8BiPlanarFullRange) {
        return;
    }

So in conclusion it might not like the fact that it's not full range, but I have no easy way to test it. Maybe you can try forcing full range if the camera and the other modules support it.

aergistal
  • 29,947
  • 5
  • 70
  • 92
  • Thanks for your help. How would I force full range? Im not a native xcode developer. – Oliver Jones Sep 11 '17 at 10:58
  • @OliverJones Sorry, me neither. The setting doesn't seem to be exposed in ARKit. Vuforia lets you set [YUV](https://library.vuforia.com/articles/Solution/Working-with-the-Camera#How-To-Access-the-Camera-Image-in-Unity) but it doesn't say what kind of YUV format it uses. And this is assuming the underlying problem is what ARKit actually reports and not a side-effect of something else (camera device not released, some other bug). – aergistal Sep 11 '17 at 11:59
0

Make sure not to use .jpg images anymore. iOS wants you to use .png images. I had the same warning/ error in Swift 4 and XCode 9 Beta.

Oscar
  • 97
  • 1
  • 12