iPhone capture session: Set custom frame-rate

Question

I've set up a captureSession and am now trying to set the framerate to 60. I am using an iPhone 12 Pro Max.

I am trying to set the frame rate with:

videoDevice?.activeVideoMinFrameDuration = CMTimeMake(value: 1, timescale: 60)

However, printing my .activeFormat tells me my iPhone only supports 30 fps.

I need 60 fps to match the frame rate of my machine learning model.

Configuration:

builtinwideanglecamera,
video,
back position,
landscape right orientation.

I do not have any cameras in this enum that allow me more than 30 fps. I create the videoDevice object therefore with:

let videoDevice = AVCaptureDevice.default(.builtInWideAngleCamera,                                           
                                          for: .video,
                                          position: .back)

What am I doing wrong?

Thanks

score 0 · Answer 1 · answered Apr 02 '21 at 14:12

videoDevice.activeFormat is only the current format. videoDevice.formats contains all the possible formats.

Mine reports many formats capable of 60fps, e.g.

<AVCaptureDeviceFormat: 0x28337d7c0 'vide'/'420f' 1280x 720, { 1- 60 fps}, HRSI:2112x1188, fov:70.291, binned, supports vis, max zoom:24.00 (upscales @1.50), AF System:1, ISO:33.0-3168.0, SS:0.000015-1.000000, supports wide color, supports multicam>
...

So pick the best format for you, then make that your activeFormat and set the desired frame durations like so:

try! videoDevice.lockForConfiguration()

videoDevice.activeFormat = my60FPSFormat
videoDevice.activeVideoMinFrameDuration = CMTime(value: 1, timescale: 60)
videoDevice.activeVideoMaxFrameDuration = CMTime(value: 1, timescale: 60)

videoDevice.unlockForConfiguration()

Another follow-up question since you an expert on this topic: What is the best way for me to test this, except for just checking if the frames collected in 3 seconds are actually 180? (My Model size) Because i literally just put 2 fps in and it still looked smooth like 30:/ — Konstantin Kuchenmeister, Apr 03 '21 at 17:45

Konstantin Kuchenmeister · Accepted Answer · 2021-04-07T07:55:20.443

Thanks, this answered my question!:)

For anyone still wondering below is the code that I used:

    // Instantiate the video device: wide angle camera, back position
    let videoDevice = AVCaptureDevice.default(.builtInWideAngleCamera,
                                              for: .video,
                                              position: .back)
    
    
    // Set the frame rate to 60, as expected by the model
    try! videoDevice?.lockForConfiguration()
    
    videoDevice?.activeFormat = (videoDevice?.formats[30])!
    videoDevice?.activeVideoMinFrameDuration = CMTimeMake(value: 1, timescale: 60)
    videoDevice?.activeVideoMaxFrameDuration = CMTimeMake(value: 1, timescale: 60)
    
    videoDevice?.unlockForConfiguration()

    // Debug only
    // print(videoDevice?.activeFormat)

However, make sure to add some error handling:D

Thanks, again.

score 0 · Answer 3 · answered Sep 18 '22 at 17:34

Though other answers provide the outline for the solution, what worked for me when I need the best resolution for the given frame rate is this. If you want to further limit certain frame resolutions, you can add a filter method to ensure a threshold on dimensions.

extension AVCaptureDevice {
    func set(frameRate: Double) {
        do { try lockForConfiguration()
            activeFormat = formats.sorted(by: { f1, f2 in
                 f1.formatDescription.dimensions.height > f2.formatDescription.dimensions.height && f1.formatDescription.dimensions.width > f2.formatDescription.dimensions.width
            }).first(where: { format in
                format.videoSupportedFrameRateRanges.contains { range in
                    range.maxFrameRate == frameRate
                }
            }) ?? activeFormat
            guard let range = activeFormat.videoSupportedFrameRateRanges.first,
                  range.minFrameRate...range.maxFrameRate ~= frameRate
            else {
                print("Requested FPS is not supported by the device's activeFormat !")
                return
            }
            activeVideoMinFrameDuration = CMTimeMake(value: 1, timescale: Int32(frameRate))
            activeVideoMaxFrameDuration = CMTimeMake(value: 1, timescale: Int32(frameRate))
            unlockForConfiguration()
        } catch {
            print("LockForConfiguration failed with error: \(error.localizedDescription)")
        }
    }
}

Kakhi Kiknadze · Answer 4 · 2023-03-05T18:14:22.640

Solution marked as the answer seemed to be not working for me but after some time I found out I was setting the frame rate before adding the device as an input. Make sure you add the device first and then set the frame rate. Otherwise it would override it with a default settings and your frame rate will be ignored. It would also be ignored if you choose a new preset for the session.

Choosing a new preset for the capture session also resets this property to its default value.

iPhone capture session: Set custom frame-rate

4 Answers4