In order to stitch the images together you need to have a homography. In order to get that homography you either have to
1. rotate your camera stationary or
2. take pictures of a planar scene
Since you take aerial shots from different positions you go for point number 2. You have to fly high so that the height of the objects is very small in relation to the flying height, so you can say you have a good approximation to a planar scene. It is recomended to take nadir shots, i.e. shots that are taken towards the centre of the earth.
If you use the stitcher class set the warper to ''PlaneWarper'' for your aerial shots. The Surf featuresfinder with these parameters works fine, at least for me:
Stitcher stitcher = Stitcher::createDefault(false);
stitcher.setWarper(new PlaneWarper());
stitcher.setFeaturesFinder(new detail::SurfFeaturesFinder(1000,3,4,3,4));