2

I'm trying to learn how 360 degree videos currently available inside VR headsets such as Oculus or Samsung Gear VR work. Apparently they are using the notion of panoramic videos (360 degree video). I was doing some tests, and noticed the raw video is larger in resolution, and inside VR a special region (which is forward facing the eyes) is initially showed which is inside the viewport. User can also change the view as desired.

Here is a sample of the raw 360 panoramic video: enter image description here

And here how inside a VR headset such as Gear VR it is seen (the major viewport): enter image description here

Now my question is that if there is any ways to detect which part of the raw video is in fact currently inside the viewport? I mean, can we detect (almost) what part of a 360 video is now being seen? Are there any API that VR headsets such as Gear VR, Oculus, Unity3D, or any other VR-related tool provide to know that, maybe using some sensors or other info?

I have limited information, so any kind of information in this regard is useful for me.

Tina J
  • 4,983
  • 13
  • 59
  • 125

1 Answers1

1

If you mean can the player detect which portion of the sphere is in the users line of sight view, then the answer is yes as the player actually has to perform transformation on the raw video to create the view that the user is actually looking at at that moment.

It does this by taking inout from the device sensors, or the key pad if it is being displayed in a browser for example, and determining from this where the user is looking at any given moment.

Based on this the player will apply transformations to the raw video to display in the screen only the portion of the video which should be in the users line of sight.

If you are asking can the server which is 'serving' the video know where a users line of sight is at any given time, then the answer is no as it simply serves the full 360 raw video (for most implementations at this time anyway).

It would be possible for the player to report back analytics to the sever indicating the users view and reporting any changes, but this would be dependent on each player individually. There are definitely companies already highlighting the ability to provide 360 video analytics in partnership or building on specific players.

Mick
  • 24,231
  • 1
  • 54
  • 120
  • Cool. So overall, the whole video is streamed and the player (at the client side) decides which portion to show given sensors and others. The thing is that I'm now checking out the 360 videos with the Gear VR and whenever I turn my head, I initially see a lower quality video which quickly improves. Doesn't it mean they are somehow communicating with the server to report back the user's view?! – Tina J May 20 '16 at 17:10
  • If it is a real time generated video then possibly, but if it is just 360 degree video then you are possibly just seeing an effect caused by the video pipeline (the set of frames being prepared and being displayed) having to suddenly flush and fill up again with the new view. The algorithms may be clever enough to do a rough pass first and then fill in more detail (not sure - the effect you mention is interesting so hopefully someone else can confirm). As an aside, because the eye is so close and screen magnified most current 360 video on headsets has pretty grainy quality as you probably saw. – Mick May 20 '16 at 21:55