Before we get much deeper into Unreal Engine... What are Mitch, Carlos, (and we) solving for?
One of the biggest issues for working in VR is Motion/Simulation Sickness.
How is it caused?
Sensory conflict theory believes that sickness will occur when a user's perception of self-motion is based on incongruent sensory inputs from the visual system, vestibular system, and non-vestibular proprioceptors, and particularly so when these inputs are at odds with the user's expectation based on prior experience. en.wikipedia.org/wiki/Virtual_reality_sickness
Five typical causes of Motion/Simulation Sickness in VR 1. Non-forward movements • No unnatural movements 2. Awareness of Vection • When a large part of the visual field moves, a viewer feels like he has moved and that the world is stationary 3. The feeling of accelerations 4. Too much camera YAW 5. Helped by adding a static reference frame Read more about it
Things we CAN DO in Unreal Engine to improve VR Games and Experiences
You MUST maintain framerate For the VR experience to feel smooth, your game needs to run on 75 hz (Oculus DK2) or even 90 hz. (HTC Vive and Oculus CV1) depending on the device. To see the current framerate type in “stat fps” or “stat unit” (for more detailed breakdown) in your console when running the game.
Use UE4’s VR Performance Profiling Tools To capture a single frame with GPU timings press Ctrl+Shift+, or type in “profilegpu” in the console. This command dumps accurate timings of the GPU, you will find that certain processes are a heavy burden on the framerate (Ambient Occlusion is one common example) when using VR. The GPU Profiling & Performance and Profiling docs are a good place to learn about profiling your game.
VR Instanced Stereo Can Help The latest 4.11 release introduces Instanced Stereo Rendering, check the video below for a comparison video of how that works. “Basical y, we’re utilizing hardware instancing to draw both eyes simultaneously with a single draw cal and pass through the render loop. This cuts down render thread CPU time significantly and also improves GPU performance. Bul et Train was seeing ~15 – 20% CPU improvement on the render thread and ~7 – 10% improvement on the GPU.” – Ryan Vance. To enable this feature in 4.11 and above, go to your Project Settings and look for “Instanced Stereo” under the Rendering category.
Disable Heavy Post-Processors Due to the demanding requirements of VR many of the advanced Post Processing features that you normally use should be disabled. To accomplish this you will need to do the following in your level. •Add a Post Process(PP) volume to your level if there is not already one there. •Select the PP volume and in the Post Process Volume section enable the Unbound option so that the settings in the PP volume will be applied to the entire level. •Expand the Settings of the Post Process Volume and then go through each section and disable any active PP settings by enabling that property by clicking on it and then set the value from the default, usually 1.0, to 0 to disable the feature. •When doing this you will not need to hit every section and set all the properties to 0. Instead first disable the really heavy hitting features like Lens Flares, Screen Space reflections, Temporal AA, SSAO, and anything else that might have an impact on performance. •While a lot of the features are disabled by setting things in your .INI this ensures that nothing will happen to performance if someone deletes the .INI by mistake.
Things to keep in at the front of your mind: LOD's and aggressive culling are a must to ensure that you are hitting your VR performance targets.
Known issues and possible workarounds: Parallax Mapping Parallax mapping takes Normal mapping to the next level by accounting for depth cues, Normal mapping does not. A Parallax mapping shader can better display depth information, making objects appear to have more detail than they do. This is because no matter what angle you look at, a Parallax map will always correct itself to show you the correct depth information from that view point. The best use of a Parallax map would be for cobblestone pathways and fine detail on surfaces.
UE4 – Lighting for VR Dimmer lights & colors can help reduce simulation sickness. Use Static Lighting over Stationary or Dynamic. Make sure your Stationary / Dynamic Lights do not overlap. Baked lighting is the best option for VR. If using Dynamic Shadows only have one shadowing light. Use Stat LightRendering to see current lighting cost. Profile, Profile, Profile
Fake shadows Wherever You Can!! Using things like fake blob shadow drop to simulate dynamic shadows are a good general rule in order to keep VR project running at frame. Blob shadow example. Image by Eric Chadwick
UE4 – Effects for VR Mesh based VFX work the best for VR. Camera Facing particles do not hold up well in VR on their own. The Dither Temporal AA Material Function can make Opacity masked objects look like Translucent ones. Local Space rotation does not look correct in VR.
UE4 – Environments for VR Reflection probes instead of screen space reflections. Again… Texture Blob shadows are a cheap alternative to dynamic shadows. The ** Merge Actor Tool ** can help cut down on Static Mesh draw call without having to do work outside of UE4.
Some very important things we all need to know about Unreal Engine.
The Unreal Engine Framework
The Unreal Engine Framework GameInstance GameMode PlayerState Pawn Class Class HUD GameState Class Class PlayerController Class
The Unreal Engine Framework The GameMode is the definition of GameInstance the game. ● It should include things like the game rules and win GameMode conditions. ● It also holds important information about: PlayerState Pawn Class ○ Pawn Class ○ PlayerContoller HUD GameState Class Class ○ GameState PlayerController Class ○ PlayerState
The Unreal Engine Framework The Pawn class is the base class GameInstance of all Actors that can be controlled by players or AI. ● The Pawn represents the GameMode physical location, rotation, etc. of a player or entity within the game. PlayerState Pawn Class ● A Character is a special type Class of Pawn that has the ability to HUD GameState Class Class walk around. PlayerController Class
The Unreal Engine Framework A PlayerController is the interface GameInstance between the Pawn and the human player controlling it. ● The PlayerController decides what to do and then issues commands to the Pawn (e.g. GameMode "start crouching", "jump"). ● Putting input handling or other functionality into the PlayerController is often PlayerState Pawn Class necessary. Class ● The PlayerController persists HUD GameState Class Class throughout the game, while the PlayerController Pawn can be transient. Class
The Unreal Engine Framework The GameInstance is a class who’s GameInstance state persists switching of levels, game modes, pawns etc. Where classes like GameMode or GameMode PlayerController are being reset and data stored in those classes is removed. PlayerState Pawn Class Class HUD GameState Class Class PlayerController Class
The Unreal Engine Framework The GameState contains the state GameInstance of the game, which could include things like the list of connected players, the score, where the GameMode pieces are in a chess game, or the list of what missions you have completed in an open world game. PlayerState Pawn Class Class HUD GameState Class Class PlayerController Class
The Unreal Engine Framework A PlayerState is the state of a GameInstance participant in the game, such as a human player or a bot that is simulating a player. Non-player AI GameMode that exists as part of the game would not have a PlayerState. PlayerState Pawn Class Class HUD GameState Class Class PlayerController Class
The Unreal Engine Framework The HUD is the base object for GameInstance displaying elements overlaid on the screen. Every human-controlled player in the game has their own GameMode instance of the AHUD class which draws to their individual Viewport. PlayerState Pawn Class Class HUD GameState Class Class PlayerController Class
Object Base building blocks in the Unreal Engine
Object Base building blocks in the Unreal Engine Actor Any object that can be placed into a level
Object Base building blocks in the Unreal Engine Actor Any object that can be placed into a level Pawn Subclass of Actor and serve as an in-game avatar
Object Base building blocks in the Unreal Engine Actor Any object that can be placed into a level Pawn Subclass of Actor and serve as an in-game avatar Subclass of a Pawn that is intended Character to be used as a player character
Pawn 1-to-1 Relationship Controller
Pawn 1-to-1 Relationship Controller CharacterMovementComponent CapsuleComponent Character SkeletalMeshComponent Etc. SpectatorPawn DefaultPawnMovementComponent Default Pawn StaticMeshComponent CollisionComponent, Etc. VehicleMovementComponent Wheeled SkeletalMeshComponent Vehicle PhysicsHandle Etc.
How about programming interactivity for VR?
Programming VR Interaction with Blueprints Blueprints in Unreal Engine is a complete visual scripting system based on the concept of using a node-based interface to create interactions from within Unreal Editor.
UE4 – Audio for VR Ambient Sound Actors in VR Ambient Sound Actor can be used for many purposes such as ambient looping sounds and non-looping sounds. Generally, the Ambient Sound Actor conforms to the real world where the closer you are to a sound, the louder it wil appear.
UE4 – Audio for VR Sound Properties You can assign a sound asset from the Details panel by selecting an asset from the Sound settings drop- down menu or by highlighting a sound asset in the Content Browser and clicking the button.
UE4 – Audio for VR Attenuation Properties Attenuation is the ability of a sound to decrease in volume as the player moves away from it. It is advisable to use Sound Attenuation objects whenever possible, if for no other reason than to give broad control over the settings for many Actors.
UE4 – Audio for VR New: Stereo Spatialization 3D spatialization is now possible for stereo audio assets. The 3D Stereo spread parameter defines the distance in game units between the left and right channels and along a vector perpendicular to the listener-emitter vector.
UE4 – Audio for VR Audio Volume Audio Volumes allow you to control and apply various sounds in your level as well as provide an avenue to create compartmentalized audio zones where you can control what is heard inside and outside of the volume.
Additional toolsets in Unreal Engine to enhance VR: Complete state of the art suite of AI Tools.
Additional toolsets in Unreal Engine to enhance VR: Complete set of tools for animation retargeting
So What’s Next?
Come tomorrow at 2 P.M. and see Unreal Engine in action.