1

I am currently working on a people detection and counting project. It basically detect for any people in the scene via USB webcam, then count people passingby. Currently, my setup is:

  1. OpenCV 2.4.6, detect people head using Haar method (floating point processing)
  2. ARM board with ARM A9 quad core and Mali quad core GPU

Unfortunately, the processing time is not fast enough 70 - 100 ms per frame (14 - 10 fps) so that people walking in normal speed or faster is not counted. The bottleneck is in the OpenCV HaarDetection method, basically 90% of the processing time per frame is consumed by the process.

I tried using another model beside Haar, the LBP model which is based on integer processing, but so far my LBP model is not satisfying and I am still working on to create new models. Also, I tried using TBB with OpenCV (multithreading natively implemented in OpenCV) but somehow cause crash in Odroid, the application works stable if I do not use TBB.

The only optimization I can think of is to utilize the Mali GPU in the board, recompiling OpenCV with modified HaarDetection to utilize some GPU processing power. My question is, is this doable using the OpenGL library? I see most example of OpenGL is to render graphic, not processing images.

genpfault
  • 51,148
  • 11
  • 85
  • 139
bonchenko
  • 567
  • 1
  • 9
  • 21
  • 3
    Most of the Mali quad cores out there today are the Mali-400 MP which only has support for OpenGL ES 2.0 and no GPGPU api to speak of. You could with a lot of willpower and time probably hack something together using the vertex/fragment shaders but it seems hardly worth it (you'd have to not only rewrite the algorithm but rewrite it in a very awkward partitioned way). Throw hardware at it or tweak your algorithm/change your approach. – PeterT Aug 22 '13 at 03:17
  • You can try using latentsvm detector, but this means training a new model for people which is not straightforward. Tell me if you're interested and I'll provide some more details. – GilLevi Aug 22 '13 at 08:46
  • Never mind, there is a trained model person. – GilLevi Aug 22 '13 at 09:13

2 Answers2

2

Other optimizations which you may consider:
1. Play with parameters - even small changes of scale factor and minimum windows size can make your algorithm faster.
2. Try to use different cascade
3. Try to play with OpenCV building parameters - WITH_TBB might help you (http://www.threadingbuildingblocks.org/) if you processor support multithreading and cascade can use more than one thread(i think that it's possible - maybe not all the time, but at least some parts of it can). Take a look at ENABLE_SSE and ENABLE_SSE2 as well.
4. Search for some other implementations of haar cascade detector or try to make it on your own - it's possible to make it faster, see(article and comments): http://www.computer-vision-software.com/blog/2009/06/fastfurious-face-detection-with-opencv/
5. If you are analysing image sequences check whether two consecutive frames are the same/very similar - if so you can skip analysis of current frame, because results will be the same(or very similar). I've used this solution in my BSc thesis(simple eyetracker using 720p webcam) and it worked fine.
6. As above + search only in regions in which difference occurs.
7. Divide your image on for example 16 rectangles. Check differences between current and previous frame in each rectangle - if all rectangles from one row or column are almost the same as in previous frame - don't analyse this row/column(pass only part of your image to haar cascade - use ROI). It should give quite good results and increase speed, because people will walk/run/etc from one side of frame to another - there is small chance that all rectangles will change between two consecutive frames.

cyriel
  • 3,522
  • 17
  • 34
  • Great! Will look forward to try those new ideas. Anyway I do quick try and also searching: 3. WITH_TBB will cause my application crash in my ARM board, but ok in PC. Debugging with Valgrind stopped with kernel error. SSE and SSE2 is applicable only in Intel system, yes? I use ARM board 5. I used motion detectior based on background subtraction, the method is not applicable is whole region is filled with moving people. Not to mention the background subtraction and connected components consumes additional time. Anyway, your suggestion leaves lots of things to try and inspires me. Thanks! – bonchenko Aug 23 '13 at 02:07
  • I follow your answer, detect on some areas based on skin colour, tune parameter, and the fps went from 15 to 50! great answer, thanks so much – bonchenko Aug 26 '13 at 03:52
0

You can try detecting people using latensvm detector (detection by parts). Luckily, there is a trained model for person here:

https://github.com/Itseez/opencv_extra/tree/master/testdata/cv/latentsvmdetector/models_VOC2007

It will probably be faster then HOG.

Hope that helps.

GilLevi
  • 2,117
  • 5
  • 22
  • 38
  • I have tried HOG before in my host computer, but it is 10x slower than Haar. But reading in some source [link](http://answers.opencv.org/question/12124/latentsvm-detector/) and [link 2](http://answers.opencv.org/question/8733/opencv-latentsvm-detector-too-slow/) states that the OpenCV implementation is not for real time processing. Never tried LatentSVM though, will try it soon. Thanks! – bonchenko Aug 22 '13 at 10:04
  • Provided GitHub link is not available please update. – Mahavirsinh Padhiyar Nov 27 '19 at 10:15