0

I am new to iOS development and currently working on an iOS app for a project where I want to detect and recognize text in an image. I am taking the image from the AVFoundation framework and getting a UIImage.
For the text detection and recognition I decided to use the Google MLToolkit, therefore I simply put the UIImage into the specific MLKit function:

func runTextRecognition(with image: UIImage) {
        let visionImage = VisionImage(image: image)
        textRecognizer.process(visionImage) { features, error in
            self.processResult(from: features, error: error)
        }
    }

Now I have the problem, that when I am taking a picture having the iPhone in portrait mode I do not get any results out of the MLKit. Whereas, when I am holding the phone in landscape mode (homebutton to the right) I get the text which is shown on the image as my result.
Is there any parameter I have to set to make Google MLKit work for portrait mode pictures? Probably I would suggest the error to be in the CameraController.swift file where I define my capture output.
You can find the repository under: https://gitlab.com/lukas.kl/TextScan.
I hope you can help me, thank you in advance!
Kind regards, Lukas

Lukas
  • 1
  • 1
  • Thanks for trying ML Kit, I took a glance at the code in TextScan, just want to confirm that the image orientation has been set correctly before passing the image to the recognizer: https://developers.google.com/ml-kit/vision/text-recognition/v2/ios#2_prepare_the_input_image – Julie Zhou Jul 25 '22 at 14:33
  • 1
    @JulieZhou Thank you very much for your reply. I also checked the image orientation and it solved the problem, that the text is now recognized in portrait mode. Now I run into the issue that the bounding boxes of the blocks, lines and elements are kind of rotated by 90° to the left. I was wondering if it has something to do with the transformMatrix function, but until now I could not solve the issue. Do you have any idea for that? – Lukas Jul 31 '22 at 06:35
  • Yeah, the sample app only handles the Portrait mode, landscape mode need more customization for the rotation, basically a mapping from the mode of the screen to the orientation of the image. – Julie Zhou Aug 01 '22 at 14:38
  • Thank you very much for your help. I just got the solution by transforming the frames of the blocks, lines and elements by changing x and y as well as the height and the width of the frames. – Lukas Aug 02 '22 at 12:10

0 Answers0