slow frame rate when rendering cifiltered ciimage and MTKView while using face detection (Vision and CIDetection)

Question

I have an app which does real time filtering on camera feed, i'm getting each frame from camera and then do some filtering using CIFilter and then pass the final frame(CIImage) to MTKView to be shown on my swiftUI view, it works fine, but when i want to do face/body detection, real time, on camera feed, frame rate goes down to 8 frames per second and super laggy. i tried anything i could find on the internet, using vision, CIDetector, CoreML, everything is the same result, well, i would do this on global thread, which makes the UI responsive but the feed which i'm showing into the main view is still laggy, but things like scrollview are working fine. so i tried to change the view from MTKView to UIImageView, Xcode shows its rendering at 120FPS (which i dont understand why, its 30FPS when not using any face detection) but the feed is still laggy, cannot keep up somehow to the output frame rate, i'm new to this, i dont understand why is it like that. i also tried just to pass the coming image to MTKView (without any filtering in between, with face detection) also the same laggy result, without face detection, it goes to 30FPS (why not 120?). this is the code i'm using for converting sampleBuffer to ciImage

extension CICameraCapture: AVCaptureVideoDataOutputSampleBufferDelegate {
  func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {

    guard let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else { return }

    var ciImage = CIImage(cvImageBuffer: imageBuffer)

    if self.cameraPosition == AVCaptureDevice.Position.front {
        ciImage = ciImage.oriented(.downMirrored)
    }
    ciImage = ciImage.transformed(by: CGAffineTransform(rotationAngle: 3 * .pi / 2))
    ciImage = ciImage.transformToOrigin(withSize: ciImage.extent.size)
    detectFace(image: ciImage) // this is for detecting face realtime, i have done it in vision 
        //and also cidetector - cidetector is a little bit faster when setted to low accuracy 
       //but still not desired result(frame rate)

    DispatchQueue.main.async {
        self.callback(ciImage)
    }

  }
}  

and this is the MTKView code, which is very simple and basic implementation of it:

import MetalKit
import CoreImage

class MetalRenderView: MTKView {
    //var textureCache: CVMetalTextureCache?

  override init(frame frameRect: CGRect, device: MTLDevice?) {
    super.init(frame: frameRect, device: device)

    if super.device == nil {
      fatalError("No support for Metal. Sorry")
    }

    framebufferOnly = false
    preferredFramesPerSecond = 120
    sampleCount = 2
  }

  required init(coder: NSCoder) {
    fatalError("init(coder:) has not been implemented")
  }

  private lazy var commandQueue: MTLCommandQueue? = {
    [unowned self] in
    return self.device!.makeCommandQueue()
  }()

  private lazy var ciContext: CIContext = {
    [unowned self] in
    return CIContext(mtlDevice: self.device!)
  }()

  var image: CIImage? {
    didSet {
        renderImage()
    }
  }

  private func renderImage() {
    guard var image = image else { return }
    image = image.transformToOrigin(withSize: drawableSize) // this is an extension to resize 
          //the image to the render size so i dont get the render error while rendering a frame

    let commandBuffer = commandQueue?.makeCommandBuffer()
    let destination = CIRenderDestination(width: Int(drawableSize.width),
                                          height: Int(drawableSize.height),
                                          pixelFormat: .bgra8Unorm,
                                          commandBuffer: commandBuffer) { () -> MTLTexture in
                                            return self.currentDrawable!.texture
    }

    try! ciContext.startTask(toRender: image, to: destination)

    commandBuffer?.present(currentDrawable!)
    commandBuffer?.commit()
    draw()

  }

}

and here is the code for face detection using CIDetector:

func detectFace (image: CIImage){
    //DispatchQueue.global().async {

            let options = [CIDetectorAccuracy: CIDetectorAccuracyHigh,
                           CIDetectorSmile: true, CIDetectorTypeFace: true] as [String : Any]

            let faceDetector = CIDetector(ofType: CIDetectorTypeFace, context: nil, 
                                          options: options)!

            let faces = faceDetector.features(in: image)

            if let face = faces.first as? CIFaceFeature {

                AppState.shared.mouth = face.mouthPosition
                AppState.shared.leftEye = face.leftEyePosition
                AppState.shared.rightEye = face.rightEyePosition
            }

    //}
}

what I have tried

1) different face detection methods, using Vision, CIDetector and also CoreML(this one not very deeply as i dont have experience in it) I would get the detection info, but frame rate is 8 or at the best case its 15 (which would be a delayed detection)

2) I've read somewhere that it might be result of the image colorsapce so i have tried different video setting and different rendering colorspace, still no change in the frame rate.

3) I'm somehow sure that it might be regarding to pixelbuffer release time, so i deep copied the imageBuffer and pass it to the detection, beside some memory issues it went up to 15 FPS, but still not minimum 30FPS. in here i also tried to convert imageBuffer to ciimage and then render ciimage to cgimage and the back to ciimage to just release the buffer, but also could not get more than 15FPS (well on average, sometimes goes to 17 or 19, but still laggy)

i'm new in this and still trying to figure it out, i would appreciate any suggestions, samples or tips that could direct me to a better path of solving this.

update

this is the camera capture setup code:

class CICameraCapture: NSObject {
    typealias Callback = (CIImage?) -> ()
    private var cameraPosition = AVCaptureDevice.Position.front
    var ciContext: CIContext?
    let callback: Callback
    private let session = AVCaptureSession()
    private let sampleBufferQueue = DispatchQueue(label: "buffer", qos: .userInitiated)//, attributes: [], autoreleaseFrequency: .workItem)
    // face detection
    //private var sequenceHandler = VNSequenceRequestHandler()
    //var request: VNCoreMLRequest!
    //var visionModel: VNCoreMLModel!
    //let detectionQ = DispatchQueue(label: "detectionQ", qos: .background)//, attributes: [], autoreleaseFrequency: .workItem)

    init(callback: @escaping Callback) {
        self.callback = callback
        super.init()
        prepareSession()
        ciContext = CIContext(mtlDevice: MTLCreateSystemDefaultDevice()!)
    }

    func start() {
        session.startRunning()
    }

    func stop() {
        session.stopRunning()
    }


  private func prepareSession() {
    session.sessionPreset = .high //.hd1920x1080
    let cameraDiscovery = AVCaptureDevice.DiscoverySession(deviceTypes: [.builtInDualCamera, .builtInWideAngleCamera], mediaType: .video, position: cameraPosition)
    guard let camera = cameraDiscovery.devices.first else { fatalError("Can't get hold of the camera") }
    //try! camera.lockForConfiguration()
    //camera.activeVideoMinFrameDuration = camera.formats[0].videoSupportedFrameRateRanges[0].minFrameDuration
    //camera.activeVideoMaxFrameDuration = camera.formats[0].videoSupportedFrameRateRanges[0].maxFrameDuration
    //camera.unlockForConfiguration()
    guard let input = try? AVCaptureDeviceInput(device: camera) else { fatalError("Can't get hold of the camera") }

    session.addInput(input)

    let output = AVCaptureVideoDataOutput()
    output.videoSettings = [:]
    //print(output.videoSettings.description)
    //[875704438, 875704422, 1111970369]
    //output.videoSettings = [kCVPixelBufferPixelFormatTypeKey as String : Int(kCVPixelFormatType_32BGRA)]
    output.setSampleBufferDelegate(self, queue: sampleBufferQueue)

    session.addOutput(output)
    session.commitConfiguration()
  }
}