MlKitAnalyzer for Android CameraX is slow for text-recognition

Question

I am implementing the new MlKitAnalalyzer based on the description and links of CameraX 1.2 found here.

Following the example I made an analyzer for Barcode scanning, and this works very well. But if I try to implement a text-scanner with MlKitAnalyzer the processing gets very slow. It takes over 4 seconds for each frame to be processed. Same code but with a manual implementation of ImageAnalysis.Analyzer processes a frame in about 600ms. Any help as to what is making MlKitAnalyzer so slow with text-recognition would be nice!

My code for creating the analyzer. Time for each frame: >4000ms:

private fun clearAndSetAnalyzer() {
    cameraController?.clearImageAnalysisAnalyzer()
    
    val startMs = SystemClock.elapsedRealtime()
    val scanner = TextRecognition.getClient(TextRecognizerOptions.DEFAULT_OPTIONS)

    val analyzer = MlKitAnalyzer(
        listOf(scanner), 
        ImageAnalysis.COORDINATE_SYSTEM_ORIGINAL, 
        mainThreadExecutor()
    ) { result ->
        result.getValue(scanner)?.let { resultText ->
            Log.i(TAG, "Scan complete net. Time: ${SystemClock.elapsedRealtime() - startMs}ms.")
        }
    }
    
    cameraController?.setImageAnalysisAnalyzer(mainThreadExecutor(), analyzer)
}

The old code for creating imageanalyzer myself is here. Initializing the scanner for every frame takes no time but is just here for illustrating what the code does. Time for each frame: 600ms:

class AnalyzerOld() : ImageAnalysis.Analyzer {

override fun analyze(imageProxy: ImageProxy) {
    val mediaImage = imageProxy.image ?: return
    val image = InputImage.fromMediaImage(mediaImage, imageProxy.imageInfo.rotationDegrees)
    val tekstScanner = TextRecognition.getClient(TextRecognizerOptions.DEFAULT_OPTIONS)

    tekstScanner.process(image)
        .addOnSuccessListener { resultText ->
            //Handling result
        }
        .addOnFailureListener { LogUtils.e(TAG, "Fail", it)  }
        .addOnCompleteListener {
            tekstScanner.close()
            imageProxy.close()
        }
    }
}

I noticed you start your measurement before you initialise your scanner. Which means, you measure the time to initialise the TextRecognition client and time to get result together aggregated. Scanner needs to be initialised only once. I think it would make more sense to move initialisation out of your measurement. Initialisation generally takes a lot of time. — Orcun, Oct 27 '22 at 09:49
@Orcun thank you for the reply. this gives output like 4005ms, 8040ms, 12101ms... The trend is clear that the time is about 4 seconds on my phone. Otherwise I agree but its a quick and dirty implementation to give general idea of time spent — Gober, Oct 31 '22 at 10:03

score 0 · Answer 1 · answered Oct 28 '22 at 02:40

0

I think your image is too large make it slow to analyze it so try to scale bitmap down or specify image resolution

You can specify image resolution here https://developer.android.com/training/camerax/configuration#specify-resolution

val imageAnalysis = ImageAnalysis.Builder() .setTargetResolution(Size(1280, 720)) .build()

Or you can scale down bitmap by using createScaledBitmap. You can check more detail from here and here

answered Oct 28 '22 at 02:40

Thành Thỏ

173
1
8

The image resolution is set automatically by LifecycleCameraController. Resolution will be same for the two analyzers so I do not think it explains the difference in processing time. Resolution seems to be 640x480 – Gober Oct 31 '22 at 10:45

score 0 · Answer 2 · answered Oct 30 '22 at 10:59

I am a new contributor to the stackoverflow platform, so my answer will be more a personal thinking about your concern. I have implemented a one-shot text-scanner in Java using the ML Kit text-recognition module and it works very well. I have tried to implement a real-time scanner. It's awfull, the text-recognition process interfere with the jerky preview, so we can't get a stable exploitable image! It's different of a barcode scanner because there are 3 prepositioned reference points on the image and the decoding process is much faster. I don't know Kotlin programming but I can identify in a short snippet the algorithm, the main instructions and their meaning. In your post it's written: "Any help as to what is making MlKitAnalyzer so slow with text-recognition would be nice!".
I read your code several times and I understand your request. I don't see how you can act on the execution time of the "scanner" process. To me, only the MLKitAnalyzer designers can answer your question. Of course I assumed that the faster "manual" implementation of the text-recognition process is functionnaly identical to the "resultText" one.

MlKitAnalyzer for Android CameraX is slow for text-recognition

2 Answers2