0

I implemented an application using ORB-SLAM2. Currently I have an OpenCV JavaCameraView in my activity and converting the camera frames to Matand send them to SLAM System. Everything is now working but I don't know how to render and show a virtual object in correct place on the camera screen.

I want to implement a simple AR app that shows some simple texts on the screen as virtual object. Any help to give me steps to achieve this will be welcome.

Ahmad Behzadi
  • 1,006
  • 15
  • 30
  • If you want basic text drawing, you can use a canvas retrieved from the camera surface view and draw the text once you have the detected coordinates. What is your exact difficulty? – yakobom Jan 26 '17 at 05:42
  • @yakobom, Currently I am only able to run the SLAM System and I don't know what should I gather from it, when user want to add a new virtual text and I don't know how to calculate the x and y of that text to be drawn on that canvas you've said when the tracking thread is running. – Ahmad Behzadi Jan 26 '17 at 08:23

1 Answers1

0

What you have do to is:

  1. From ORB-SLAM you have a point cloud of 3D points. You have to place your virtual object in this point cloud according to where you want it to be (meaning you assign x,y and z coordinate to the points of the virtual object)
  2. Then you have to project these 3D points onto the image plane by using the projection matrix specific to your camera, which will give you the positions of the 3D points on the 2D image plane
  3. Then you have to draw your object/ points on the image according to the 2D positions you got from (2).

ORB-SLAM itself has also a functionality to do simple AR, maybe you could also have a look at the code they use for it.

Silvan
  • 1
  • 2
  • Yeah, I've seen the AR sample. But couldn't convert it to android. All the functions were for Linux libraries. – Ahmad Behzadi Apr 23 '17 at 07:08
  • Currently I'm able to get All 3D point and project them into screen. Now the problem is that how can I convert somewhere that user touched on the 2D screen to a 3D point? (You know the sample AR source code in original ORB_SLAM is creating only a plane exactly under the camera.) – Ahmad Behzadi Apr 23 '17 at 07:16
  • I am also working on the same thing right now. Are you open to using Unity? I have used it on iOS successfully - placed an object in 3D space that I can move around. – Faraz Hassan May 04 '17 at 14:29