0

I'm using the front-facing camera on an iPhone X to capture depth data (using the IR true depth camera).

I'm able to quite easily extract the depthData from the photo that I take, but that data appears to be limited to 8-bit values per data point, even though I believe I am asking for 32-bit precision. See below:

func photoOutput(_ output: AVCapturePhotoOutput, didFinishProcessingPhoto photo: AVCapturePhoto, error: Error?) {    
    let depthData = photo.depthData

    if (depthData != nil) {
        print("got some depth data")

        let depthData = (depthData?.converting(toDepthDataType: kCVPixelFormatType_DepthFloat32))
    }
}

The depthData object in the snippet above contains only 8-bit values. Is there greater precision available from the AVDepthData?

Vadim Kotov
  • 8,084
  • 8
  • 48
  • 62
Vik
  • 1,301
  • 4
  • 16
  • 29

1 Answers1

2

I haven't tried, so I'm not sure if you can get a "deeper" depth format out of the TrueDepth camera.

If you can, however, converting the depth data that comes out of the capture isn't the way to do it. depthData.converting(toDepthDataType:) is analogous to converting scalar types. For example, if you have a value of type Float, the extra decimal places you gain by converting it to Double are all zeros — your existing measurement hasn't gained any precision.

The way you specify depth capture formats is before capture. Set your capture device's activeDepthDataFormat to one of the supportedDepthDataFormats compatible with its current activeFormat. The values you find in supportedDepthDataFormats will tell you what types / precision of depth data your capture device is capable of recording.

rickster
  • 124,678
  • 26
  • 272
  • 326
  • Great, I implemented setting the activeDepthDataFormat before capture as you suggested. So far, it _appears_ to be returning more precise data. I chose this format: 'dpth'/'fdep' 640x 480, { 2- 30 fps}, HRSI: 640x 480, fov:56.559 This was based on this (https://forums.developer.apple.com/thread/81795). The above format was the last one in the array of supportedDepthDataFormats, so I chose the last item in the array. @rickster, do you know how I can programatically set to this format more intelligently than that? i.e. how do I say, set to format("fdep") or something similar. – Vik Feb 24 '18 at 19:20
  • 1
    Glad it worked! You can’t construct AVCaptureDeviceFormat objects, if that’s what you’re asking. The whole point of the supported-formats array is being able to choose only from those formats the device claims to support. Instead, iterate or filter the array, selecting whichever one meets all your requirements (to do that you may need to inspect the associated CMFormatDescription using the global functions related to that type. – rickster Feb 24 '18 at 22:40
  • for absolute accuracy, do you know if the data at each pixel is in 1 / meters? Or is it straight up meters? – Vik Feb 25 '18 at 17:22
  • Disparity formats are always 1/m, depth formats are meters. – rickster Feb 25 '18 at 17:49