Android Camera2 Renderscript overheat issue

Question

I have this overheat issue, that it turns off my phone after running for a couple of hours. I want to run this 24/7, please help me to improve this:

I use Camera2 interface, RAW format followed by a renderscript to convert YUV420888 to rgba. My renderscript is as below:

#pragma version(1)
#pragma rs java_package_name(com.sensennetworks.sengaze)
#pragma rs_fp_relaxed

rs_allocation gCurrentFrame;
rs_allocation gByteFrame;
int32_t gFrameWidth;

uchar4 __attribute__((kernel)) yuv2RGBAByteArray(uchar4 prevPixel,uint32_t x,uint32_t y)
{

    // Read in pixel values from latest frame - YUV color space
    // The functions rsGetElementAtYuv_uchar_? require API 18
    uchar4 curPixel;
    curPixel.r = rsGetElementAtYuv_uchar_Y(gCurrentFrame, x, y);
    curPixel.g = rsGetElementAtYuv_uchar_U(gCurrentFrame, x, y);
    curPixel.b = rsGetElementAtYuv_uchar_V(gCurrentFrame, x, y);

    // uchar4 rsYuvToRGBA_uchar4(uchar y, uchar u, uchar v);
    // This function uses the NTSC formulae to convert YUV to RBG
    uchar4 out = rsYuvToRGBA_uchar4(curPixel.r, curPixel.g, curPixel.b);

    rsSetElementAt_uchar(gByteFrame, out.r, 4 * (y*gFrameWidth + x) + 0 );
    rsSetElementAt_uchar(gByteFrame, out.g, 4 * (y*gFrameWidth + x) + 1 );
    rsSetElementAt_uchar(gByteFrame, out.b, 4 * (y*gFrameWidth + x) + 2 );
    rsSetElementAt_uchar(gByteFrame, 255, 4 * (y*gFrameWidth + x) + 3 );

    return out;
}

This is where I call the renderscript to convert to rgba:

@Override
        public void onBufferAvailable(Allocation a) {
            inputAllocation.ioReceive();

            // Run processing pass if we should send a frame
            final long current = System.currentTimeMillis();
            if ((current - lastProcessed) >= frameEveryMs) {
                yuv2rgbaScript.forEach_yuv2RGBAByteArray(scriptAllocation, outputAllocation);
                if (rgbaByteArrayCallback != null) {
                    outputAllocationByte.copyTo(outBufferByte);
                    rgbaByteArrayCallback.onRGBAArrayByte(outBufferByte);
                }
                lastProcessed = current;
            }
        }

And this is the callback to run image processing using OpenCV:

@Override
    public void onRGBAArrayByte(byte[] rgbaByteArray) {

        try {
            /* Fill images. */
            rgbaMat.put(0, 0, rgbaByteArray);

            analytic.processFrame(rgbaMat);

            /* Send fps to UI for debug purpose. */
            calcFPS(true);
        } catch (Exception e) {
            e.printStackTrace();
        }

    }

The whole thing runs at ~22fps. I've checked carefully and there is no memory leaks. But after running this for some time even with the screen off, the phone gets very hot, and turn off itself. Note if I remove the image processing part, the issue still persists. What could be wrong with this? I could turn on the phone camera app and leave it running for hours without a problem.

Does renderscript cause the heat?
Does 22fps cause the heat? Maybe I should reduce it?
Does Android background service cause heat?

Thanks.

ps: I tested this on LG G4 with full Camera2 interface support.

how long are you keeping on the camera? Because the camera hardware is seralizing 24 frames PER SECOND to a video. If you continously run this over time, that is a lot of Disk IO that would generate heat. Sounds logical to me. — JoxTraex, Mar 30 '17 at 04:29
Camera, RenderScript, OpenCV, plus potential IO and network usage, could easily push the phone to its thermal limit. One thing you could try is to use a lower resolution, or lower fps, or attach an heat sink to it :) — Miao Wang, Mar 30 '17 at 18:46

score 2 · Answer 1 · answered Mar 30 '17 at 19:51

In theory, your device should throttle itself if it starts to overheat, and never shut down. This would just reduce your frame rate as the device warms up. But some devices aren't as good at this as they should be, unfortunately.

Basically, anything that reduces your CPU / GPU usage will reduce power consumption and heat generation. Basic tips:

Do not copy buffers. Each copy is very expensive when you're doing it at ~30fps. Here, you're copying from Allocation to byte[], and then from that byte[] to the rgbaMat. That's 2x as expensive as just copying from the Allocation to the rgbaMat. Unfortunately, I'm not sure there's a direct way to copy from the Allocation to the rgbaMat, or to create an Allocation that's backed by the same memory as the rgbaMat.
Are you sure you can't do your OpenCV processing on YUV data instead? That'll save you a lot of overhead here; the RGB->YUV conversion is not cheap when not done in hardware.
There's also an RS intrinsic, ScriptIntrinsicYuvToRgb, which may give you better performance than your hand-written loop.

Android Camera2 Renderscript overheat issue

1 Answers1