Sadly, Meta killed Facebook 360 Spatial Workstation - one of the best FREE tools to
do immersive 3D Spatial Audio for VR video games, 360 and VR180 videos on Oculus TV and YouTube VR.
This tutorial is a must if you need a reliable solution moving forward to mix 3D Spatial Audio.
If you just got the Canon VR180 lens or the new Insta360 camera, the next step is to learn how
to do spatial audio. If you want to release your VR180 or 360 videos in the Meta version of the
Metaverse, it requires to have spatial audio. So there is no way to avoid it. This tutorial series,
will teach you step by step to master 3D Spatial Audio with the latest immersive tools. Let's go!
Hey, what's up, everybody? It is your Boi Hugh here. If you are looking for a FREE solution
to do spatial audio. The best way is still the Facebook 360 Spatial Workstation. Since you can
no longer download it from the official website, here is the latest version from my Google Drive.
I will continue hosting the download for the community. If you need a tutorial on how to
use it, please refer to my tutorial here. If you need help on YouTube VR release with
Headlock audio support, check out this tutorial here.
This tutorial, I will teach you the best alternative I found,
which is also the most affordable and complete. It includes a binaural preview,
a VR headset plugins for head tracking mixing, a spatial limiter, and a spatial reverb plugin.
You can use it on any host - including ProTools, Reaper, Adobe Premiere, and Adobe Audition.
In this first tutorial, we will focus on the professional audio engineer workflow,
which is using a DAW - Digital Audio Workstation.
In the next tutorial, we will learn how to mix it directly inside an NLE like Adobe
Premiere or Davinci Resolve. So don't forget to subscribe and hit the bell.
Even tho you can technically do everything inside Adobe Premiere,
I won't skip this tutorial. Understanding the basics is very important,
especially if you are a filmmaker, not an audio engineer.
Preparation Here we have our final
locked VR180 edit shot on the Canon R5C with the Canon Dual Fisheye lens. Remember,
you don't start spatial audio workflow until you can get a picture lock from your client.
Here I go ahead and lock my edit. This is a live concert. We got
individual mic output as audio stems from the venue audio engineer. As you see here,
we have the left Volin track and the right Cello track. Also, we have a stereo mix here.
This is your headlock audio. If you are running your own audio, remember to get timecode to have
timecode sync. I have an in-depth tutorial here on how to get individual sound sources
using 32-bit audio and jamming the timecode into your Canon R5C or your VR camera of choice.
Why having a timecode is important? Well, let me show you. Select all the audio and right click,
synchronize, select Audio and hit okay. As you see, the actual syncing is still off!
So don't just clap and hope for the best. Here we need to fix that. Right click here the
hamburger bar, check "show audio time units". This enables less than a frame, very fine adjustment.
Listen and look at the waveform here, align that waveform to your best ability like so.
Okay, now all audio stems are synced. We are going to export them one by one. You
can just use waveform audio as output format. In this simple setup, we have left Volin and
right Cello, and the Headlock Stereo mix. Now, we need to prepare the video. We don't
need 3D for visual reference. So if you have a top and bottom 3D 360 video shot
on Insta360 Pro 2 or Titan, use VR Projection effect to convert them into mono 360.
Here we have left and right VR180. We are first gonna render out the final release video.
We use H.265, 5760 by 2880 in resolution, and 60Mbps. Set the VR to stereo left and right,
180. Uncheck the audio as we don't need them. Now go ahead and render.
Now, we need to create a mono 360 reference video for the DAW. Most of the DAW do not
understand 180 videos, so it is easier to render them out as 360 to standardize the process.
Change the sequence settings to Monoscopic and 360. Now drop in VR Projection Effects,
it should convert your VR180 to mono 360. Control M to render the full sequence again.
You also do not need high resolution here. So I only use 1920 x 960 resolution, H264 will be fine,
or you can use ProRes Proxy. If you want smooth in VR headset playback later on
inside your DAW, make sure the resolution is as low as it can be.
Okay. Now we have everything. You can close Premiere and open up your DAW. We are going
to use Reaper here. Reaper is FREE and that is why we use it here. But if you find it useful,
please consider paying for a license to support the amazing developers. It is very affordable.
The next thing you need is Noisemakers Ambi Bundle HD and Ambi Eye. Noisemakers is not free,
but it is one of the best professional plugins suits and happens to be the cheapest. It has Ambi
Verb and Ambi Limiter, necessary audio plugins that the FREE Facebook Audio Workstation lacks.
So go ahead and download a free version from Noisemakers website and follow along.
Open Reaper and create a new project. First, drop in the proxy 360 video we just
rendered into Reaper. Again, this video should be in small resolution H.264 or ProRes Proxy codecs.
Don't inject H.265 or ProRes 422 into Reaper. If you don't see the video windows, go to View,
and select Video. Immediately turn off the scratch audio as we
don't need them. In repear, just like another DAW, for example, Ableton - it is more fun to control
it with a control surface or Midi controller. I have the Monogramcc here - which I use it to
control Premiere and Color grading in Resolve. But it can easily program into Midi controller for my
Reaper. Speed is everything in post production. Now import the stereo audio for Headlock
audio mix. Headlock audio mix is optional.
For narrative VR films - it is recommand to use Headlock audio as background music and VO.
I pull down the volume to minor 6DB as your headlock track will always be too loud in the mix.
Start with - 6DB and drop the volume even more as needed. For live music or VR music video,
we don't need headlock audio, so I just mute it for simplity.
Now drop in the individual audio stems - the Voline and Cello you just rendered from Premiere.
Remember, each sound source is a new track. A VR film usually has 20 to
all the way 100+ tracks. As each sound effects need to have its own track.
Go ahead and hit the routing here on the violin track, we need to make this into 16-channel
3rd-order ambisonic. 16 channels will cover both Meta Quest release and YouTube VR. The goal here
is to mix once and work on all platforms. Do the same thing with Cello to turn it into 16 channels.
Now go ahead and click FX button right here and add Noisemaker Ambi Pan VST plugin
into both Violin and Cello tracks. If you click this little eye icon here show
a transparent overlay here. You can move it and overlay on your video windows exactly like so.
Try to play it, and it does not sound right. Because you can not hear Spatial audio without
first turning it into binaural. Let's create a new track and name it Ambi Master. We are going to
route all ambisonic tracks into this Ambi Master. Go back the routing of each track,
turn off master send here and add a send to Ambi Master instead. Make sure your audio send is
multiple channels 1 to 16 as we are send ambisonic to this ambi Master. Now do the same with Cello.
Click FX on Ambi Master and add Noisemakers Ambi Head HD. Click this gear icon on the top right
of the plugin to turn on settings. Here we target the Facebook 360. If you are releasing on YouTube
and want to hear what your mix sound like on YouTube without uploading on YouTube first,
choose YouTube 360 here. This is actually why I will use Noisemakers instead of the Free
Facebook Audio Spatial Workstation. It allows me to mix based on platforms as
Google has different spatial audio compared to Meta. This is a truly a one size fit all
solution. Your input here is 3rd Order ambisonic. 4 channel is 1st order, and 16 channel is 3rd
order. YouTube is 1st order and Meta is close to 3rd order. Just remember that you should be fine.
Now if you open back up Ambi Pan on the Violin and move the dot around your head,
you hear the audio travel around your head - like 8D audio on YouTube. Very cool effects on its own.
That is actually how to create 8D audio, if you are into that kinda of thing.
Now, I will color each sound source and rename it - it is important to stay organized when you
have hundreds of soundtracks. Click the gear icon here on the Ambi Pan.
First, the input should change to Stereo as we input a Stereo track instead of a Mono track.
VO is usually mono, and music is usually stereo. I change the color to yellow and named it Violion.
Add Ambi Pan on Cello as well. Change it to Stereo,
change the color to blue and name it Cello. Now, use the panel here and the overlap panel
as a reference to place your sound sources. Here is the direction, and here is the height of the
audio. Violin is higher than Cello here so we need to place it to make the environment believable.
Now listen to it, use the ambi head on the Ambi Master track to pan your virtual manikin head. You
can head the cello is pretty low no matter what direction you turn your head. So we increase the
volume on the Cello by about 4 DB, and we drop the violin - which is way too loud - to about
-1.5DB. Now the mix is a lot more balance. It sounds like Game of Throne, which is pretty cool.
It is okay to just mix with Ambi Head. But if you want the full immersion and mix inside VR headsets
like the Meta Quest 2, especially clients are on set. We will try a special player from Noisemakers
called Ambi Eye. Ambi Eye is free so go download it on their website.
Ambi Eye is just like Adobe Premiere, it use SteamVR to connect your Meta Quest
2 or any VR headset to your computer. So you need first to install SteamVR. After
SteamVR is open and you go into Oculus Link - neither use a USB-C cable like I did here,
or use Air Link. Now, you can run Ambi Eye. It should detect your Meta Quest 2.
Go ahead and open the same video file, go to Reaper, and hit play,
it should sync inside your Meta Quest 2. Looking around will head what the final mix sounds like.
After we hear the mix inside Meta Quest 2 - we spot several problems. The biggest
problem is the audio clip when you look directly at the instrument. Since we love
the loudness level right now, we can add a spatial limiter to prevent anything from
clipping. Go to Ambi Master track, add Ambi Limiter as an effect and make sure it
go before the Ambi Head. All effects has to be before Ambi head. Let's limit the output to -1db,
so nothing will be higher than -1db no matter where viewers are looking.
Now the second problem is the audio is very dry and sound recorded in a studio instead of an
actual concert hall. There should be spatial reverb in an actual physical environment.
So let's give viewers this real immersion by adding spatial reverb. Add Ambi Verb on Ambi
Master track, move it before the limiter so the limiter can help final limit everything.
Play it.
This is way too much. Choose a different room.
I personally absolutely love the philharmonic concert hall reverb. But we still don't want to
over do it, so we turn the decay all the way down to 10, wet level to 30 - keep it subtle.
Really nice! Now we have a professional mix, it is time to render it out and release on Meta Quest 2.
First, go to Ambi Master and turn off Ambi Head effect entirely. This
is only for mixing preview but not for the final render. Remember that this is important.
While Ambi Master is selected, go to file - render - selective track and entire project.
Pick you location and name, make sure manually type in 16 here at channels, and hit render.
You should see 16 channels are going. If not, something is wrong. Go back to check your routing.
You can also select the headlock audio and render it separately.
Named it HL_master and changed it back to stereo. Render that as well.
Now we are going to mux video and audio together to release on Meta Quest 2 or YouTube VR. Open up
FB360 Encoder. FB360 Encoder is part of the old Facebook Spatial Workstation. But it is continue
supported by Meta and you can download it free on the Meta website. For now, you can also just
download it directly from my link in the YouTube description as that is the most stable version.
Open FB360 Encoder, select Facebook 180 video as this is a VR180 video shot on the Canon. The
format we have is B-format 3rd Order AmbiX. Remember, 16-channel is 3rd order. HL Stereo
is optional. But if you need it, you drop your stereo HL render file here. Video layout is left
and right stereoscopic. 360, you choose top and bottom stereoscopic or mono. Don't touch
the Focus thing, and hit render. The final file can be upload directly onto Meta Oculus Media
Studio. Follow this tutorial on how to release your content onto Oculus TV for the world to see.
The Same file can be uploaded on DeoVR as well.
DeoVR is a third party player and it supports the same order ambisonic audio as official Meta Quest.
For YouTube VR, select YouTube Video with 1st order ambisonic. Still drop the same 16-channel
file here under B-format 3rd Order AmbiX. Do not use HL audio as YouTube currently does
not support HL Audio. If you need HL audio, the hack is to route the HL Audio into the 1 channel
of the 4-channel ambisonic. That is a tutorial for other time. Pick
Left and right stereo and render. The final file can be upload directly on YouTube as VR180 video.
Okay, this is a pretty long but very necessary tutorial.
I am sure you have ton of questions if you are new to audio. Don't be shy and ask your questions
in the comment session down below. I will do my best to help you out. DM me on Instagram
too if I did not get back to you in time. I did not cover how to animate your audio
in Reaper, b/c I already covered that in my original tutorial here. So go check that out.
The next tutorial will be transferring everything you just learned today and into
Adobe Premiere and Adobe Audition. We will do all the mixing inside native Premiere,
or Davinci Resolve instead of Reaper.
If you like more tutorials like this,
don't forget to subscribe and give this video a thumb up. We work really hard to cover every
single immersive filmmaking topic so you have everything you need to create immersive films
for the Metaverse. To continue to support this channel, so you have free knowledge and
feel like you have superpower in VR, share this video on your social media and help us to grow.
I will see you in the next tutorial or our next original VR release on the Meta Quest.