Becoming a VR engineer: Part 1 — Cinematic VR overview
Series explaining what you need to know to become a VR engineer
Cinematic VR
This is a summary of a class provided by https://purplepill.io/
You can find the class from here
What you need to create a cinematic VR
- 360 degree 3D camera
- ambisonic microphone: record sound field
360 camera categories
- 360 2D
- 360 stereoscopic 3D: adding depth by using inter-pupillary distance ~64mm for a typical adult
- 360 algorithmic 3D
- 360 light field
Needs stitching
Glossary
6-dof (six degrees of freedom)
The ability to move inside a virtual reality experience over the 3 rotational axes, as well as the 3 translational axes. In other words; you can move your head (pan, tilt and roll) and your body (forward/backward, up/down, left/right).
Ambisonic Audio
A full-surround audio format which captures a complete sphere of sound information. Ambisonic microphones output in four-track A-format, which should then be decoded to B-format for editing and distribution.
Augmented Reality (AR)
Superimposing a virtual layer over a user’s view of the real world. So in essence a mix between reality and computer-generated content, and can be seen as an enhancement of the real world with virtual information.
Binaural audio
Binaural audio uses an HRTF to take the shape of your head into account, which creates a realistic listening experience where you can clearly hear the direction of sounds. Binaural audio only works with headphones, not over speakers.
Cinematic VR
A branch of virtual reality that covers high-quality, 360° 3D video experiences, preferably with spatial audio, and possibly with interactive elements.
Control Points
With control points you essentially tell the stitching software that this pixel in image #1 is the same as a that pixel in image #2. This way the software knows better how to stitch the individual images together into a seamless 360° panorama.
Equirectangular Projection
An image projection used for cinematic VR content, which applies a certain distortion to a panorama so that it is shown correctly when projected on the inside of a sphere. Also commonly used to display flat images of world maps.
Field of View (FOV)
The degrees of vision in a virtual reality headset. The bigger the Field of View (FOV), the more you’ll be able to see in one glance. FOV is also used to describe the area which a camera captures.
First-View
The first part of a panorama that is seen by a viewer. Often the same as front-view, unless a viewer is not looking straight ahead when the scene starts.
Frames per Second (FPS)
The number of unique images are shown per second in a video. Regular video is usually played back at 30 FPS, but for a smooth cinematic VR experience at least 60 FPS is required.
Front-View
The part of a panorama that is seen by a viewer in their initial position. So the position when the viewer first puts on a VR headset and looks straight ahead.
Haptic Feedback
Technology to simulate the sense of touch by applying forces, vibrations and motions to the user. A haptic feedback glove would in theory enable you to grab something in the virtual world and actually feel you are holding it.
Head Mounted Display (HMD)
Screens you wear on your head, like virtual reality headsets.
Head Related Transfer Function (HRTF)
Your ears receive sounds, but first the shape of your head interferes and distorts these sounds. HRTFs models the shape of a head and distorts the sounds accordingly to create a ultra-realistic binaural listening experience.
Interpupillary Distance (IPD)
The distance between the center of the pupils of two the two eyes. This distance varies per person, but is on average 64mm for adult males. Important for creating a realistic 3D effect.
Latency
The delay between input and response. In the case of VR, the delay between moving your head and the movement of the image in the VR headset. Latency is a prime cause of simulator sickness and therefore a major focus point for companies like Oculus.
Light Field Camera
These cameras, which use thousands of tiny lenses, capture not just pixels, but also the angle in which light rays are travelling. This allows images to be refocused after they have been taken. The first commercial light field cameras have been developed by Lytro.
Live-Action
Films with real people instead of computer graphics. Most cinematic VR content falls into this category, and is therefore sometimes referred to as Live-Action VR.
Mixed Reality (MR)
Games and experiences that combine objects from the physical world and the virtual world. So AR enhances the real world, while in MR the real world also influences the virtual world.
Nadir
The point directly below an observer, so the bottom of your cinematic VR content. The bottom of your video usually requires some cleaning up, as it most likely shows either a black spot or your tripod. This process is called nadir patching.
Parallax
The apparent change of an object’s position when looked at from a different angle. Significant parallax effects occur when people or objects are too close to a 360 video camera, making it nearly impossible to get a clean stitch.
Presence
The degree to which your brain believes you are present in a particular place. Virtual reality tries to trick your mind into believing you are actually somewhere else.
Refresh Rate (Hz)
The number of times per second a display updates its content. The refresh rate is expressed in Hertz. So a refresh rate of 60Hz means the display updates its content 60 times per second. The Oculus consumer edition will function at 90Hz, or 90 frames per second.
Resolution
The amount of pixels an image is composed of. In cinematic VR, ultra high resolutions are required, since images, and therefore the resolution, is spread out over a sphere.
Simulator Sickness
A form of motion sickness which can occur after extended exposure to virtual reality content. It is also commonly experienced by pilots undergoing flight simulator training for extended periods of time.
Shutter Sync
The synchronized opening and closing of the shutters of several individual cameras.
Stereoscopic 3D
A technique to add an illusion of depth to a flat image by using two cameras aimed at the same object, but offset by the interpupillary distance. By showing one image to the left eye, and the other (offset) image to the right eye, our brains perceive depth.
Stitching
Using software to combine several individual images into a seamless panorama.
Stitchline
A visible seam in a panorama image, caused by imperfect stitching.
Virtual Reality (VR)
A computer-generated reality which can be viewed in a head mounted display. The goal of VR is to transport the user to a virtual place and make them feel they are actually present there.
Zenith
The point directly above an observer. So the top of your sphere in a cinematic VR setting.
