7

I'm writing a program that stitches aerial images from a video in real time taken by a drone. In fact, what I'm doing is:

  1. get two consecutive frames
  2. find features between these two frames
  3. calculate homography
  4. calculate images offset and stitch them together

The problem is the 3rd step, and that's why: Both my code and opencv stitching function calculate homography asserting that I've a rotation in some axis (because it's a panorama). Actually, the drone is acting like a "scanner", so instead having a rotation I've a translation of the camera. This influences my homography and then my stitching step creating artifacts. There's a way to consider the camera translation instead a rotation (like orthographic thing)? If it helps, I've intrinsic camera parameters calculated with the calibration tutorial

Ps: I'm programming in c++

EDIT I've found this library written by Nasa http://ti.arc.nasa.gov/tech/asr/intelligent-robotics/nasa-vision-workbench/

could it help? Can I use it togheter with opencv?

bjorn
  • 338
  • 6
  • 18
  • if your features are correct and well distributed, homography computation should work. There are no assumptions like "camera is rotating" ... but: homographies are only correct for either planar scenes or rotating cameras (but often gives ok results for nearly planar scenes). – Micka Jun 12 '15 at 08:33
  • can you adf sample images and result? did you undistort your lens? – Micka Jun 12 '15 at 08:34
  • 1
    Yes, I've undistort my camera lens with the opencv utility. here's the flith of my drone:https://mega.co.nz/#!u1AQBJAD!GZpUYTXBn9ZWLLIxirOndXZCh3LvQsC5AYtUHok11vU I don't know why but frames are stitched wrongly after a certain point. I'm extracting feature with AKAZE and matching them with 2_nn – bjorn Jun 12 '15 at 09:10
  • in stitching there is some kind of (afaik unavoidable) drift. depending on your framerate it might be better not not stitch each frame to reduce the drift. your link doesn't work for me... – Micka Jun 12 '15 at 10:04
  • My video has a framerate of 30fps. For now I'm taking 1 frame at second. I tried the image link and it works! – bjorn Jun 12 '15 at 10:08
  • ah the link just didnt work in browser mobile mode... works in desktop version but cant unpack 7zip files on my mobile device... sorry... – Micka Jun 12 '15 at 10:40
  • did you try bundle adjustment techniques to reduce drift? there it might be possible to use known information like intrinsics and translation/rotation guesses – Micka Jun 12 '15 at 10:41
  • No I didn't try bundle adjustment.. I've found this paper http://cdn.intechopen.com/pdfs-wm/47643.pdf and they don't do adjustment and I'm following the same step (except for the thing they implemented) but I can't really figure out why it isn't working :S – bjorn Jun 12 '15 at 10:58
  • one more thing: your "planar" images are in fact kind of 'spherical' (earth radius) instead of planar, right? – Micka Jun 12 '15 at 12:39
  • Well there's a distortion caused by the lens (I think the camera is a gopro) so I undistort the frame and then it should be planar.. Or am I wrong? – bjorn Jun 12 '15 at 13:53
  • your lens/image must be undistorted thats right, but the "scene" must be planar if you want to use homographies without artifacts (if your camera translates). for an aerial image with high distance, the ground is locally near planar. but in fact, if you stitch big panoramas, the earth curvature becomes more and more important (no near-planarity anymore). – Micka Jun 12 '15 at 14:22
  • Ohhhh that's right, I understand now. The fact is that the videos I have aren't high (15 meters more or less). There's homography alternative that does not generat artifacts? – bjorn Jun 12 '15 at 15:29
  • 1
    Actually the thing that sounds strange to me is that there is no way to get a mosaic without ruin it. It's very strange that opencv does not have an orthographic projection for points! – bjorn Jun 12 '15 at 19:19
  • I see that the question is 3 years old, but, I would like to clarify, did you have a problem with calculating translation using homography? If so, then there is an easy solution, but I suspect that it is no the case – Bolat Tleubayev Apr 04 '19 at 10:29

0 Answers0