Look at all these VR180 cameras we have - the RED V-raptor 8K 120fps VR180 cinema-quality camera,
the Canon R5 with Canon RF Dual Fisheye, another Canon R5 with more dual fisheye lens...
As we continue to innovate in VR storytelling, negative comments are also coming in. Most of
them surrounding the topics on "3D VR180 is not truly VR". I don't completely
disagree with that. 6 degrees of freedom is part of the evolution of XR Video in the Metaverse.
On one hand, we need to stay true as a filmmaker to capture reality as real as it comes without
computer-generated game engine graphics. On the other hand, we need to provide viewers the freedom
to move around in the metaverse we capture. This is a very difficult, and expensive task to tackle.
Today, I am happy to tell you, we have some significant progress to share.
Stop calling VR180 is not true VR. We now have the ability to transform any VR180
video into 6DOF experience using WebXR on Meta Quest 2 and other comparable VR headsets.
In this video, we will explain why 6DoF is better from a viewer's perspective.
We will introduce a new AI technology to transform 3DoF video into 6DoF using depth
mapping. I will show you a hands-on workflow using Canon R5 and the Canon
RF Dual fisheye lens to capture, process, and release your own WebXR
6DoF experience. And I will show you how to share and enjoy it in your Meta Quest 2.
We got a lot to cover, let's jump into the Metaverse now.
Hey, what's up, everybody! It is your boy Hugh here from the Metaverse. The technology I am going
to show you now, which is developed by Lifecast VR, is still in very early development. Yes, there
will be lots of artifacts. But also, it is very cool. Let's me just show you and stop talking.
You can see the same result as well on your Oculus Quest 2 VR headset.
Go to your Oculus Browser, type in one of these 3 URLs. Make sure you use HTTPS instead of HTTP
for security reason. You should see this page. Just hit the play icon, the video will play as
180 video on YouTube. But when you hit this "Enter VR" button, wow!
There are lots of artifacts. But at the same time, this is a true 3D 6DoF experience. I can move my
head side to side and the image responds. The artifacts are more obvious when I move in a large
distance, but in general, I feel less motion sick compared to the traditional 3DoF VR180 experience.
Here is a still 3D 180 photo capture with the Canon R5 and dual fisheye lens.
Here is it turns into a 6DoF experience. It is kinda like a NFT - a very comfortable
blended 3D 180 art. Where I can even walk around to see Keeley break down into pieces.
Keeley: " Hey, be nice! "
So this will support both the photo and video capture with the Canon
EOS VR system. But any VR180 camera will work. You don't necessarily need to use
Canon. As you see this example is captured with the RED V-Raptar in slow-motion.
If you don't understand why 6DoF is better than 3DoF in the Metaverse, let me explain.
6 degrees of freedom, aka 6DoF, is the motion system most game engine-based VR experience built
upon. Most of the VR180 or 3D Stereoscopic 360 videos are 3DoF - 3 degrees of freedom.
In 3DoF, a user can rotate their head, and their view responds, but if they move side to side,
their view does not respond. This can potentially cause motion sickness because
the user's inner ear perceives motion, while their eyes don't see the same motion. This motion
sickness is on a subconscious level. But this is one of the causes of motion sickness in VR.
With 6DoF experience, what you see within VR responds correctly to all motion - Forward &
backward, up & down, left & right, yaw, pitch & roll. There is no more disconnection between
what you see and what your subconscious tells you. This makes the 6DoF experience,
in general, more immersive and comfortable. Furthermore, 3DOF formats produce
incorrect stereoscopic views in VR if the user rolls their head,
or looks up or down from the horizon, which can cause eye strain. 6DOF fixes these issues as well.
Creating 6DoF videos is harder than
3DoF because it requires technology like depth map, photogrammetry,
volumetric capture, or light-field reconstruction of each frame of the video. There is a lot
of research on this topic in academia and big tech companies like Meta Facebook reality lab,
Google, HTC you named it. But very few practical solutions are available to VR filmmakers and
regular viewers like me and you. Microsoft Metastage by Christina Heller, RED Manifold,
and my friend Tobias Chen's VolCap camera system are some of the very expensive early adapters.
What we introduce here is an AI-software conversion by a little startup called Lifecast.
Besides paying a very small software licensing fees, the rest can be free if you already own
VR180 camera - like the brand new Canon EOS VR system reviewed on this channel.
To go one step further, Lifecast AI solution is also working on fixing motion sickness
in VR video where the camera moves while recording. Moving VR180 camera
even on a gimbal can be problematic. 6DoF unlocks the possibility to capture 3D POV
VR video with wearable cameras, and watching them in VR feels like reliving a memory.
Here is a prototype demo of this idea by Lifecast VR
By using the power of WebXR, a decentralized web3.0 technology, we can view the 6DOF in any VR
platform that supports WebXR. Like Oculus Browser here, free within your Meta Quest 2. Even better,
for devices like mobile phones or web browsers that don't support VR - you can view a fallback
interactive 180 video just like on YouTube. So you can build your website for the future, but with
nice backward comparability. In the rest of this video, I will show you how to do that for free.
So go ahead, bring your VR180 camera out and go
shoot something. You don't need to defish with the Canon VR Utility.
Lifecast VR software work directly with Canon raw fisheye footage - save you 1 extra rendering step.
Here, we use the VR180 format instead of the RAW fisheye. Since my video is already edited
and ready to go. Download the Lifecast VR software from the link in the description. It has both Mac
and PC versions. PC version support Nividia GPU for AI learning - so the render is a lot faster.
Here is the software interface. You can do both images and video. We choose our video here as
input data. Output into a jpg sequence onto a folder. As you see it will generate 4 views
here - depth map and background layer. The rest I will leave it untouched.
After render is done, you can use FFMPEG to mux all the jpg into H264 for web browser and
iOs device delivery. And render an H265 version for high-quality Oculus Browser
WebXR playback. And 1080P fallback video for very old devices.
I will provide all 3 command lines for PC and MAC in the video description. Just copy and
paste. If you need some help on FFMPEG, check out my Oculus Quest render tutorial right here.
I always provide extra value and money to my viewers. This part of the tutorial,
I will teach you how to host your WebXR experience FREE, in a secure manner,
so everyone in the world can view your experience on any device. Usually, this might cost you $100
hosting fee pre-month. But you can get it free with some simple command lines
What we use is surge.sh - a static website deploying solution with HTTPS support. All
you need to do is to run the below command in your terminal if you use Mac or CMD if you are on PC.
First, type in npm install --global surge. You might need to use Sudo depending on
your Mac setup. Then go download the website example from this URL.
Use a text editor like Sublime - also free - edit the index.html.
Change this part into this part - with your video name instead.
See the video description for details.
Save your index.html Now in the terminal or CMD,
go into your website folder. Mine is under G drive. Simply type in surge, confirm the folder,
change your URL if you like - maybe call it funky chicken 222 and they hit enter.
Now your whole website is uploaded to funky-chicken-222.surge.sh
Now, put on your Oculus VR headset. Open browser and type
in https://funky-chicken-222.surge.sh or whatever URL you choose and ...
Your first 6DOF video now is out on the internet for the world to enjoy!
Thank you so much for watching this kinda in-depth, little bit nerdy,
but totally cool video. Help us to help you to stay on the bleeding edge of XR development by
liking and sharing this video. And subscribe to our YouTube channel, Instagram page and Facebook
page if you have not already. More metaverse friends and creators joining our community
help us give a bigger voice to move the industry in a positive direction - pushing real chance
in XR development for not only technologists but everyday storytellers like me and you.
As always, we love your feedback and comment. 3DoF vs 6DoF is a big discussion in XR development.
We just cover the tip of the iceberg. We need your input, the community input,
to drive our content strategy. So we can better serve you and your interests.
We are hungry for XR knowledge, I hope that is you too! Until next time, see you in the metaverse.