How to use fisheye camera in unity in Google Tango?

Question

I would like to use Google Tango's fisheye camera to take a picture of the environment then analyse on it. However, how can I achieve this in the Unity environment? Should I continue with the tango AR camera?

Thank you.

Answer 1

This is currently not support directly from Unity SDK, you can only use fisheye image from C or Java.

Answer 2

I did some experiments with the development tablet and it worked pretty well for me. You just have to dig quite deep depending on what you're doing. At several places in the Unity SDK there's TANGO_CAMERA_COLOR hardcoded, look for that and try replacing it with TANGO_CAMERA_FISHEYE. I got the fisheye working for the AR overlay after a while. The problems is that if you're doing AR, you'll miss an accurate mathematical description of the physical camera and you'll have hard time setting up the Unity cam to match the fisheye fov and distortion precisely. If you figure this out, let me know :)

Answer 3

You can create a fisheye on any platform by using 5 cameras pointing front, left, right, top and bottom, with 90 degrees FOV that render to a texture, then assemble those textures into distorted meshes that resemble the fisheye distortion into a plane. Paul Bourke made quite some research about this topic and there's a free asset he made for Unity that does exactly this.

Here's a screenshot of the (modified) demo scene from his package that shows the how the meshes are setup to create the fisheye effect.

Doing analysis on the image though is another story and it'd be convenient to do it on the single 5 snapshot by doing a numerical spherical projection rather than laying on the assembled fisheye result.