Google Cardboard video texture on iOS with Unity 5

Question

I'm trying to make a spherical 360 video with the Google Cardboard SDK.

I have made some tests with MovieTexture for a desktop build that works fine, but there is no implementation of this class in iOS.

I have seen some external plugins on the asset store, but none of them seem to be compatible with Unity 5.

Is there a class provided by the Cardboard SDK to achieve this or if not do you know of another solution or plugin that can accomplish this?

Answer 1

Movies in Unity are usually rendered as textures on objects. On mobile the issue becomes that the device only wants to display video in a video player, so the Unity class MovieTexture is not supported.

I am having success circumventing this, and successfully rendering 360-video on the inside of a sphere using a Unity plug-in from the Unity Asset Store called, Easy Movie Texture.

For working on a Mac, here's what I did:

1. Download the Easy Movie Texture plug-in from the Unity Asset Store
2. Open the Demo Sphere demo scene from Assets/EasyMovieTexture/Scene
3. Create a new (empty) Prefab to your project, and drag the Sphere GameObject from the Demo Sphere scene onto the Prefab.
4. Reopen your Cardboard scene and drag the new videosphere prefab into your hierarchy.
5. Open your source 360-video in Quicktime
6. File -> Export -> 720p
7. Change file extension from '.mov' to '.mp4'
8. Drag your new mp4 file into your projects Assets/Streaming Assets directory. Note: don't import through the menu system, as this will force Unity to convert to OGG.
9. On the "Media Player Ctrl" script component of your videosphere GameObject, locate the "Str_File_Name" field and provide the FULL filename of your newly exported video file. Make sure to include the extension as part of the string, "mymovie.mp4".

Pretty sure that's everything. Hope it helps other folks stuck on this problem.

Final note, the video will only render on the device. In the editor you will only see a white texture on the sphere. You have to publish to the device in order to see your awesome 360-video.

Answer 2

I have finally solved it.

We have to delete CardboardAppController and MMTAppController and merge them into one.

Basically in the Assets/Plugin/ios there is 2 AppControllers: "CardboardAppController.mm" (also has a CardboardAppController.h header) and "MMTAppController.mm"

The trick is that ios only "see" one app controller after generation. Since CardboardAppController comes first, MMTAppController is discarded.

To solve this issue I have done the following:

Create new AppController: in my case it was named SVAppController.mm and SVAppController.h

Then we have to merge both app controller into that one and delete them.

So I copy pasted the content of CardboardAppController.h into SVAppController.h and the content of CardboardAppController.mm into SVAppController.mm (Make sure to change the names in the code replace CardboardAppController by SVAppController)

Then copy past the content of MMTAppController.mm into SVAppController.mm in the right place

And finally delete CardboardAppController and MMTAppController.

You will be left with 1 app controller and everything should be working fine.

For reference Here is my final SVAppController:

SVAppController.h

#import "UnityAppController.h"
#import "UnityAppController+Rendering.h"
#import "UnityAppController+ViewHandling.h"

// Unity 4.6.2 added a category to the app controller.
#if UNITY_VERSION < 462
#import "UnityInterface.h"
#else
#import "UnityAppController+UnityInterface.h"
#endif

// Unity 4 used a different method name to create the UnityView.
#if UNITY_VERSION < 500
#define createUnityView initUnityViewImpl
#endif

@interface SVAppController : UnityAppController

- (void)preStartUnity;

- (UnityView *)createUnityView;

- (void)startSettingsDialog:(UIViewController *)dialog;

- (void)stopSettingsDialog;

- (void)pause:(bool)paused;

- (void)shouldAttachRenderDelegate;

@end

SVAppController.mm

#import "SVAppController.h"
#import "CardboardView.h"
#import <UIKit/UIKit.h>
extern "C" {

    extern void readProfile();
    extern void syncProfile();

    extern "C" void MMTUnitySetGraphicsDevice(void* device, int deviceType, int eventType);
    extern "C" void MMTUnityRenderEvent(int marker);

    extern UIViewController* createSettingsDialog(id app);
    extern UIViewController* createOnboardingDialog(id app);

    bool isOpenGLAPI() {
#if UNITY_VERSION < 463
        return true;
#else
        SVAppController* app = (SVAppController *)GetAppController();
        UnityRenderingAPI api = [app renderingAPI];
        return api == apiOpenGLES2 || api == apiOpenGLES3;
#endif
    }

    void launchSettingsDialog() {
        SVAppController* app = (SVAppController *)GetAppController();
        [app startSettingsDialog:createSettingsDialog(app)];
    }

    void launchOnboardingDialog() {
        SVAppController* app = (SVAppController *)GetAppController();
        [app startSettingsDialog:createOnboardingDialog(app)];
    }

    void endSettingsDialog() {
        SVAppController* app = (SVAppController *)GetAppController();
        [app stopSettingsDialog];
    }

}  // extern "C"

@implementation SVAppController

- (void)preStartUnity {
    [super preStartUnity];
    syncProfile();
}

- (UnityView *)createUnityView {
    return [[CardboardView alloc] initFromMainScreen];
}

- (void)startSettingsDialog:(UIViewController*)dialog {
    [self pause:YES];
    [self.rootViewController presentViewController:dialog animated:NO completion:nil];
}

- (void)stopSettingsDialog {
    [[self rootViewController] dismissViewControllerAnimated:NO completion:nil];
    [self pause:NO];
}

- (void)pause:(bool)paused {
#if UNITY_VERSION < 462
    UnityPause(paused);
#else
    self.paused = paused;
#endif
}

- (void)shouldAttachRenderDelegate;
{
    UnityRegisterRenderingPlugin(&MMTUnitySetGraphicsDevice, &MMTUnityRenderEvent);
}

@end

IMPL_APP_CONTROLLER_SUBCLASS(SVAppController)