1

I have a view controller with an ARSKView whose ARSession I configure like this:

let configuration = ARWorldTrackingSessionConfiguration()
arskView.session.run(configuration)

In its associated SKScene I have this method implementation:

override func update(_ currentTime: TimeInterval) {
    guard let arskView = self.view as? ARSKView else {
        return
    }

    if let currentFrame = arskView.session.currentFrame {
       let capturedImage = currentFrame.capturedImage
       var requestOptions: [VNImageOption : Any] = [:]

       if let cameraData = CMGetAttachment(capturedImage, kCMSampleBufferAttachmentKey_CameraIntrinsicMatrix, nil) {
          requestOptions = [.cameraIntrinsics : cameraData]
       }

       let request = VNDetectTextRectanglesRequest { [weak self] (request, error) in
          self?.detectTextHandler(request: request, error: error)
       }
       request.reportCharacterBoxes = true

       let imageRequestHandler = VNImageRequestHandler(cvPixelBuffer: capturedImage, orientation: 6, options: requestOptions)

       do {
          try imageRequestHandler.perform([request])
       } catch {
          print(error)
       }
    }
}

Then, some logic to draw rectangles wrapping the detected text. I found such logic in this article, where ARKit and thus ARSKView are not used, but AVCaptureSession and AVCaptureVideoPreviewLayer instead. When you run that example, the camera has better quality and text boxes are more accurately drawn than when I run my project using ARKit and ARSKView.

I need to use ARKit with SpriteKit, but when I move the camera close to texts they are shown blurred, and in general the boxes wrapping words are drawn quite misplaced.

Is there any way I could improve this?

Andy Jazz
  • 49,178
  • 17
  • 136
  • 220
AppsDev
  • 12,319
  • 23
  • 93
  • 186

0 Answers0