4

Writing an app which takes the preview frames from camera does some transformation to it and then displays it on screen.

in

public void onPreviewFrame(byte[] data, Camera camera) {}

I take the data do yuv2rgb and some pixel manipulation in JNI in another thread. Then I create bitmap from the RGB int array and draw it using 

canvas.drawBitmap(bmp, 0, 0, null);

I get around 15-20FPS on HTC Nexus One at 640x480 and 30+ FPS on Samsung Galaxy S II

I am wondering if I could speed things up by doing the drawing using Android OpenGL ES? I would be following this guide: http://obviam.net/index.php/texture-mapping-opengl-android-displaying-images-using-opengl-and-squares/

tomi
  • 525
  • 9
  • 20

2 Answers2

4

Are you using SurfaceView for your camera App? The preview frame that you get is a copy of the actual frame that gets displayed on the screen. I am guessing you add another view to display the callback preview frames.

I am not a graphics guy but why don't you try SurfaceTexture. You can use setpreviewtexture() instead of setpreviewwindow(). This way Camera Service will send the buffers directly to the app, instead of making a copy and also queuing it to the AndroidNativeWindow. This might improve your performance.

This is how android Camera app Panorama works. You can find the source code here

vikky.rk
  • 3,989
  • 5
  • 29
  • 32
2

Implemented OpenGL ES and it does help with performance

tomi
  • 525
  • 9
  • 20
  • Hey I know this is old but maybe you can help me out? I am working on a camera app and trying to find resources on how to implement OpenGL ES to improve performance? – IZI_Shadow_IZI Feb 26 '13 at 17:17
  • Could you be more specific what you need? – tomi Feb 27 '13 at 08:16
  • Well basically I just want to implement the camera preview through opengl. Via a GLSurfaceView. I will then use that to enable the user to snap a photo. I suppose its similar to how SnapChats and Instagrams camera features work? – IZI_Shadow_IZI Feb 27 '13 at 13:31
  • basically first you must get the frames from camera, I think it's in onPreviewFrame method. Now those frames are in YUV422 format so you might probably need to convert them to RGB if you would like to do som e manipulation on it. Afterwards you send it to the OpenGl – tomi Mar 05 '13 at 12:03
  • I understand it's been a long time since you did this and you probably don't even remember the statistics but I'm still asking, how much was the improvement? – timemanx Sep 05 '13 at 19:58
  • The improvement was huge. You can check the app: https://play.google.com/store/apps/details?id=si.sis.mirrorsfree&hl=en – tomi Sep 06 '13 at 07:55
  • I've resorted to using a `SurfaceTexture` to have the frames delivered to an OpenGL texture. But I'm still getting about 22fps at a resolution of **1280x720**. Are there any tips that you could give me to get at least over 25. – timemanx Sep 06 '13 at 21:06
  • Where is the holdup? Is the camera sending enough FPS or is there some other holdup? – tomi Sep 09 '13 at 11:11
  • Yea it's the camera. It's not sending the frames fast enough. If I draw the frame only when `onFrameAvailable(SurfaceTexture s)` is called, then I get 22fps. But if I draw the frame as soon as the previous frame is done drawing, then I get more than 50 fps. – timemanx Sep 09 '13 at 15:07
  • [This](https://dpaste.de/OkJPD/) is basically it. If I remove `glSurfaceView.setRenderMode(GLSurfaceView.RENDERMODE_WHEN_DIRTY)` from `onCreate()`, then I get more than 50 fps. Otherwise, just about 22. So, it's definitely not OpenGL that's slowing things down. – timemanx Sep 11 '13 at 21:57
  • If you set RENDERMODE_WHEN_DIRY then you define when the redraw happens usually in ON_NEW_CAMERA_FRAME... But if you don't set that option than OPENGL will continuosly redraw even if it didn't get any new frames from the camera and thus give the impression of running at 50fps – tomi Sep 12 '13 at 07:01
  • Yea. So that means the camera itself is sending frames at a lower rate. Any idea why? – timemanx Sep 12 '13 at 08:18
  • 1
    probably because of this: http://stackoverflow.com/questions/8676046/why-fps-is-not-same-as-original-camera-app – tomi Sep 13 '13 at 06:43