0

I would like to know if there is a way, keep displaying camera preview at 30 fps with full camera resolution and meanwhile do some image processing on received frames. For example preview has 30fps while the image processing can only process about 15 frames per second due to some complex calculations. Is this possible keep this preview smooth and perform some image processing on the background?

For image processing I am using java and OpenCV.

Infinito
  • 85
  • 1
  • 3
  • 10
  • Only process every other frame. Or, put the frames on a work queue with a bounded size, where older queue entries get dropped to make room for newer ones, and have your image processor pull frames off the work queue. Whether you can achieve your desired FPS will depend a lot on the nature of the image processing (NDK? RenderScript Compute? Java?) and the processing power of the device. – CommonsWare Nov 24 '15 at 13:19
  • Edited main post. But when I am processing every frame I am getting about 15fps with 740x480 resolution, so this preview does not look good. That is why I would like to run preview simultaneous with image processing in the background. – Infinito Nov 24 '15 at 13:23
  • Depending on the camera API in use (version 2 or the older, deprecated version), the preview type and hardware you'll see different results. For example, using the older Camera API and the callback buffer mechanism I've seen different devices feed frames at different rates for the same resolution. And this is with *no* image processing, just straight preview images being fed. – Larry Schiefer Nov 24 '15 at 13:34
  • 1
    One way to process frames quickly is to use the GPU instead of the CPU. For example: https://www.youtube.com/watch?v=kH9kCP2T5Gg – fadden Nov 24 '15 at 16:53
  • I was thinking about it, but before I try it, I am researching if there are other ways to achieve that. – Infinito Nov 24 '15 at 22:53

1 Answers1

1

Four years ago, the question was about 15 fps, and the camera resolutions back then were not what we have today.

Your mileage will vary. What is the full resolution for your camera? What CPU do you have? Can you build your algorithms to run multithreaded? Or maybe, if latency is not an issue, you can simply jiggle the incoming frames between worker threads, typically one thread per core.

At any rate (pun intended), you need to be very polite with the camera. Here are some tips on using the old camera API correctly. Let me reproduce them here:

Community
  • 1
  • 1
Alex Cohn
  • 56,089
  • 9
  • 113
  • 307
  • Resolution of my camera is quite high (Samsung Galaxy S5), CPU is quad-core 2.5 GHz, and I am mostly using openCV algorithms which should be ran multithreaded anyway. For getting camera frames I am using CameraBridgeViewBase which is giving better results than my own class for retrieving frames. If I could achieve about 20 fps for 1280x720 I would be really satisfied. – Infinito Nov 24 '15 at 17:15
  • Check how fast is the tutorial-1 app. – Alex Cohn Nov 25 '15 at 07:23
  • If you are saying about OpenCV tutorial 1 example, okay it is fast, but it does not have much calculation comparing to my image processing, so it is hard to compare these two. – Infinito Nov 26 '15 at 08:18
  • Well, it does not spend time in image processing, but it does provide a yardstick. It can also deliver RGB for you, which does involve some image processing. The bottom line is, if OpenCV provides the raw pixels at reliable frame rate to you, but your calculations are too slow to keep up with it, you can parallelize your algorithm, either by working on frame slices independently, or by jiggling the incoming frames between worker threads. – Alex Cohn Nov 26 '15 at 09:17