MLKit Text Recognition in portrait

Question

I have been following this link regarding Firebase MLKit text recognition (OCR) for iOS and it seems to be working fine with the exception of when the photo selected (via either camera or library) was taken in portrait. When processing a photo in portrait no content is detected on the image.

I have been taking pictures from the same distance to the object and ensuring that the image is sharp and in focus.

Firebase MLKit Tutorial

Is this a limitation of on device MLKit text recognition or is there some setting i have been over looking?

Do i need to manipulate the image and rotate it? That would seem odd!

Any pointers would be greatly appreciated

score 0 · Answer 1 · answered Nov 08 '19 at 16:47

0

The image will have to be put into the recogniser the "right way up". Please refer to the iOS quick start sample for details.

answered Nov 08 '19 at 16:47

Hoi

599
2
7

1

Thanks - i actually forgot i had this question open. I got it working in the end by ensuring the orientation was right :) – teh_raab Nov 08 '19 at 16:49

score 0 · Answer 2 · answered Nov 08 '19 at 16:51

So it was due to the orientation. The solution which was grabbed from some other site..

override fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent?) {
    super.onActivityResult(requestCode, resultCode, data)

    if(requestCode == REQUEST_IMAGE_CAPTURE && resultCode == Activity.RESULT_OK) {
        if (imageUri == null) {
            Log.d(TAG, "imageUri is null!")
            return
        }
        Log.d(TAG, "$imageUri")
        val bmp = MediaStore.Images.Media.getBitmap(contentResolver, imageUri)

        //we need to rotate the image as cameras can be embedded in the hardware in different orientations.  Make the bitmap upright before we attempt to process it..
        val rotatedImg = rotateImageIfRequired(this, bmp, imageUri!!)

        processImage(rotatedImg) //parse this into the ML vision
    } else {
        Log.d(TAG, "onActivityResult -> resultCode: $resultCode")
    }
}

private fun rotateImageIfRequired(context: Context, img: Bitmap, selectedImage: Uri): Bitmap {
    Log.d(TAG, "rotateImageIfRequired")
    val input = context.getContentResolver().openInputStream(selectedImage);
    var ei: ExifInterface? = null
    if (Build.VERSION.SDK_INT > 23)
        ei = ExifInterface(input);
    else
        ei = ExifInterface(selectedImage.getPath());

    val orientation = ei.getAttributeInt(ExifInterface.TAG_ORIENTATION, ExifInterface.ORIENTATION_NORMAL);

    when (orientation) {
        ExifInterface.ORIENTATION_ROTATE_90 -> return rotateImage(img, 90f);
        ExifInterface.ORIENTATION_ROTATE_180 -> return rotateImage(img, 180f);
        ExifInterface.ORIENTATION_ROTATE_270 -> return rotateImage(img, 270f);
        else -> return img;
    }
}

private fun rotateImage(img: Bitmap, degree: Float): Bitmap {
    Log.d(TAG, "rotateImage() degree: $degree")
    var matrix = Matrix();
    matrix.postRotate(degree);
    val rotatedImg = Bitmap.createBitmap(img, 0, 0, img.getWidth(), img.getHeight(), matrix, true);
    img.recycle();
    return rotatedImg;
}

MLKit Text Recognition in portrait

2 Answers2