How to Overcome Key Challenges of Building VR Apps

There’s a sense that Virtual Reality (VR) apps are easy to develop. The general belief is that you just add an Oculus Rift plugin and your mobile app will magically transform into a VR app. Although this sounds really great, it’s far from reality.

Developing apps for VR is highly complex because the ultimate goal here is to make the user feel subconsciously present in a virtual world. This is extremely difficult to successfully execute because we humans have highly evolved brains that need to accept this virtual reality.

Adapt Existing Apps to VR

If the apps weren’t designed for VR from the beginning, it will be very difficult to pull off. You can think of it like the early days of radio and TV. Radio was essentially broadcasting opera while TV was broadcasting plays. It’s only over time that television started to incorporate camera movements, cuts, and zoom to breath new life into this visual format.

It’s the same for VR apps as mobile apps and VR apps are completely different mediums. At first, we may see a lot of brands try to modify existing apps for VR, but this has the potential to only provide a poor experience which can hurt the industry as a whole.

Latency is Critical

The biggest challenge in building apps for VR is latency. Think about it, if you turn your head and the image you’re viewing takes a second to appear, your brain will reject that the image and head movement are related.

This instability can also make the user feel dizzy, so avoiding this is key to developing a successful app. Experts suggest that 4ms end-to-end latency from the movement will be acceptable for the human brain.

Further, if you intend to run the application in stereoscopy, you have to always be aware that the 3D engine will need to compute the left and right images concurrently for each frame. Although it’s difficult to do much about display latency, you can ensure that your application runs fast. 

Complex World of 3D/VR Interactions

When it comes to VR apps, interaction is a huge challenge as the user doesn’t have access to a keyboard or mouse. What complicates it further is the fact that 3D interaction is a complicated ergonomic problem that’s difficult to overcome.

So what are the ways of interacting with VR?

• Select and manipulate an object
• Navigate within an environment
• Enter text or numbers
• Graphical User Interfaces (GUIs)

Accomplishing these tasks are possible in many different ways, but it will depend on the app, the user, and the hardware. Interaction can be achieved through voice commands, hand gestures, by physical movement (like walking), or even by using a handheld device like a joystick.

Head-Mounted Display (HMD) and Avatars

If you’re developing apps for an HMD, you have to build avatars that can completely immerse the user in a virtual world.   

If the VR systems come enabled with full body tracking, then you can easily build avatars that match the same dimensions as the user. But if the VR system has only a few trackers, then it will be difficult to match the exact dimensions on the user.

As a result, you will end up with an avatar that is either bigger or smaller than the user. But getting the avatar to work won’t be enough to build a successful VR app.

The success of VR apps will depend on collaboration as the user will want to interact with others in a virtual world. As a result, the VR application needs to be connected to a single server where all the data between applications can be synchronized securely to ensure that users see other avatars in real-time.

When you’re developing a VR app, you also need to make sure that it works as intended on different VR hardware.

No matter what kind of hardware the user is equipped with, the application should be able to do the following:

• Manage multiple graphic cards and/or screens
• Compute the correct perspective
• Display a variety of stereoscopy
• Manage multiple computers and synchronize Framelock, Swaplock, and Genlock
• Haptic force-feedback

So it all comes down to building a good VR app that does more than just utilize a stereoscopic camera and a single tracker. You also have to ensure the following:

• Low latency
• Interactions look appropriate
• Avatars interact as intended
• Collaborate efficiently
• User presence is maintained throughout the experience

Having said all this, executing a VR app that works as intended is not impossible. Further, as this is still a new field, there might be a learning curve for developers to overcome.

As the industry evolves and apps become more complex, it will be interesting to see the new challenges that come up over time.

And what has been your experience with building VP apps so far?