Getting started with Mobile VR

vr

#1

VR is awesome. I love it. We’re on the cusp of some exciting new developments across the VR stack and within the VR tech spectrum.

The Spectrum

VR tech has a broad spectrum of technologies that range from the low end (mobile VR with static 360 content) to high end (fully immersive room scale experiences for platforms like the Vive).

Understanding this when working in VR is very important, not everything is and should be treated the same. There are certainly different applications and use cases throughout the spectrum.

The Stack

The VR stack is equally diverse, varied and super complicated, thanks to the rapidly evolving nature of the technology involved.

Included in the VR stack are:

For the purposes of this thread, I’m going to explore the lower end of the spectrum, in terms of the technology options, and building real world mobile VR applications… here we go.


I welcome to you to join in and share your own experience, regardless of your own expertise or focus on any part of the stack - whether you’re a 3D and CGI guru, mobile developer, web developer, 360 / 3D content creator, game developer, maker and maintainer of tools like 360 libraries and SDKs.

Please comment on anything that’s going on in my posts, correct me when I’m inevitably wrong or misinformed and start your own posts to explore something new…


#2

Google Cardboard for iOS - VRView

A very simple way to get started with VR in your mobile app, is to use the Google Cardboard SDK and more specifically, the VRView that Google provides for rendering 360 degree images and videos in your native app views.

Here’s how you’d use it…

I’m working on an iOS project, written in Swift. The Google VRView and dependencies are written in ObjectiveC so we need to do a little setup dance to use the libraries in our project.

I used cocoapods to install the library:

Podfile

platform :ios, '9.0'
use_frameworks!

target 'VRapp' do
  pod 'GVRSDK'
end

in terminal,

$ cd ~/VRapp

$ pod install

Close Xcode and open the .xcworkspace, if this is your first Pod installation.

You’ll need to add a Bridging Header file to your project.

File > New > File

Create a new Header file.

Name it like this, replacing APPNAME with your app name…

APPNAME-Bridging-Header.h

The contents, including the Google VR View stuff, will look like this…

#ifndef APPNAME_Bridging_Header_h
#define APPNAME_Bridging_Header_h


#import "GVRWidgetView.h"
#import "GVRPanoramaView.h"
#import "GVROverlayView.h"
#import "GVRVideoView.h"
#import "GVRHeadTransform.h"
#import "GVRCardboardView.h"
#import "GVRAudioEngine.h"

#endif /* APPNAME_Bridging_Header_h */

If you still have some problems from here, you may need to go into Build Settings > Objective-C Bridging Header and make sure the path to your bridging header file is in this setting.

Install Objective-C compatibility header should be set to YES

Once the setup faff is sorted, the rest is pretty straightforward…

In your ViewController file

Step 1:
Ensure that the ViewController conforms to GVRVideoViewDelegate

...
class VideoDetailViewController: UIViewController, GVRVideoViewDelegate {
...
}

Step 2:
Add a suitable 360 video file to your project. Note whether the file is in Stereo or Mono format.

Step 3:
In ViewDidLoad()


override func viewDidLoad() {
  super.viewDidLoad()

    GVRVideoViewManager.sharedInstance.resume()
        GVRVideoViewManager.sharedInstance.frame = CGRectMake(0, 0, view.frame.width, view.frame.height)
        GVRVideoViewManager.sharedInstance.delegate = self
        GVRVideoViewManager.sharedInstance.enableFullscreenButton = true
        GVRVideoViewManager.sharedInstance.enableCardboardButton = true

       // URL to video file
        let videoPath = NSBundle.mainBundle().pathForResource("videoOne", ofType: "mp4")
        let videoURL = NSURL(fileURLWithPath: videoPath!)

      // initialise the VRView with the Video
        GVRVideoViewManager.sharedInstance.loadFromUrl(videoURL, ofType: .Mono)

    // Add it to the ViewController view stack
        view.addSubview(GVRVideoViewManager.sharedInstance)
    }


#3

Step 4

Add the VRView delegates and do what you need to here…

func widgetViewDidTap(widgetView: GVRWidgetView!) {
        // implement
    }
    
func widgetView(widgetView: GVRWidgetView!, didLoadContent content: AnyObject!) {
        // implement
    }
    
func widgetView(widgetView: GVRWidgetView!, didFailToLoadContent content: AnyObject!, withErrorMessage errorMessage: String!) {
        // implement
    }
    
func videoView(videoView: GVRVideoView!, didUpdatePosition position: NSTimeInterval) {
        // implement
    }

Memory Issues

I’m using 0.9.0 of the SDK and have had some issues with memory leaks and due to the sheer size of the VR videos that can easily be hundreds of MB or even GB sized.

One way to work with lots of instances of VRView, perhaps you have an app that presents lots of different VR videos and images, is to create a Singleton class that ensure that you only have one instance of the VRView in memory at a time, and therefore avoid the chance of keeping multiple resource draining instances in memory and inevitably, resulting in your app crashing at runtime.

You’ll notice that’s what I’m using in the code above, calling the GVRVideoViewManager.sharedInstance instead of creating a new instance of GVRVideoView() each time…

Create a new Swift file, called GVRVideoViewManager.swift

import Foundation

class GVRVideoViewManager{
    static var sharedInstance = GVRVideoView()
    private init(){}
}

This at least resolves my issue with memory leaks and crashes when loading videos into the default view, but I do still have a problem with the app crashing when I load the full screen cardboard view, in order to watch via my Cardboard VR mask…

There’s a Github issue raised and looks like the Google Dev’s are aware…