1

Note: I am not asking for code just the name of some algorithms I can research

Problem

I'm trying to do a 3D reconstruction from multiple images where the subject is rotating.

I don't have a lot of experience in the CV field and my mathematics is not particularly good so I am reliant on either simply worded papers or existing libraries/tools.

Where I'm At

I have been researching and testing using Structure from Motion (with VisualSFM) to generate a 3d point cloud and Multi View Stereo (with CMPMVS) to reconstruct the scene.

Obviously these algorithms are designed to process images where the camera (or cameras) moves and the scene is stationary so the reconstructions fail for a number of reasons.

Example: I've been working with short videos of a person rotating around a marker on the floor, see below. I've tried removing the background but this seems to make it harder for SfM to build pairs, probably because of the lack of intformation. Above: Person Rotating on the Spot, Below: Same with Background Removed

Question

Does anyone know the name of an algorithm/pipeline that will be able to reconstruct a series of images with a static background and a rotating object?

Or is it not possible to do a 3d reconstruction in this way?

hammus
  • 2,602
  • 2
  • 19
  • 37
  • 2
    in theory, it doesn't make any difference whether the object or the camera is moving/rotating. I guess your problem is the background (so some objects are moving and others don't)? Can you try to remove the background from the images or use some kind of ROI or sth.? – Micka Jan 17 '17 at 08:23
  • Check this paper and video: Paper: https://www.google.ru/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0ahUKEwjxnduL5MjRAhXIIpoKHeOJBsAQFggcMAA&url=http%3A%2F%2Fwww.bmva.org%2Fbmvc%2F2009%2FPapers%2FPaper297%2FPaper297.pdf&usg=AFQjCNHeMssZh1XIZS9O-YAkk8qk8APflQ&sig2=i1UcXDhmg_pUtCAfo9ohKw Video: https://www.youtube.com/watch?v=vEOmzjImsVc – Andrey Smorodov Jan 17 '17 at 08:41
  • @AndreySmorodov Thankyou looking at them now – hammus Jan 17 '17 at 08:56
  • @Micka Thats what I thought and have already tried removing background, (which I thought was a defacto ROI) but I suspect that maybe SfM or MVS looks for straight lines or there just simply arent enough pair matches in the input images? – hammus Jan 17 '17 at 08:57
  • What kind of texture does the rotating object have and are you able to get correspondences between consecutive frames? If you could post a couple of screenshots of the video, that would be really helpful too! – Utkarsh Sinha Jan 17 '17 at 11:00
  • @UtkarshSinha good idea, on your suggestion I have added a photo of one of the image sets Im working with. Please let me know if you have any ideas! – hammus Jan 17 '17 at 12:32
  • 1
    You images looks like dinosaur sequence here http://blogs.mathworks.com/loren/2009/12/16/carving-a-dinosaur/ , may be the carving algorithm also will be useful for you? – Andrey Smorodov Jan 17 '17 at 15:32
  • @AndreySmorodov that looks very promising! I'm going to try it now! – hammus Jan 17 '17 at 16:27
  • Please let us know if it worked! – Utkarsh Sinha Apr 22 '17 at 21:02
  • Hey @UtkarshSinha the methods works great with a large number of photo like the dinosaur sequence in AndreySmorodov s link above, which I think has around 30 example photos. if you download the resources you can recreate the dinosaur example fairly easily, it's a slightly different approach that involves Voxel layering in the beginning – hammus Apr 23 '17 at 00:10

0 Answers0