[MUSIC PLAYING]
Hi.
My name's John Escobar.
In this video, we're going to be taking a look at the Facebook Spatial
Workstation and how to use that in an ambisonic mix project.
[MUSIC PLAYING]
Let's begin by taking a look at the different plugins in the suite.
The first plugin that we should look at is the Audio 360 Spatialiser.
Essentially, this is our 360 panner.
This is the plugin that should end up on every single track of your project
that you intend to spatialize into a 360 environment.
It should also be the very last plugin that ends up on the track.
If you're going to do any other effects processing,
that should all happen prior to this plugin.
At the top left corner, there's an Input section.
This Input section allows us to tell the plugin what kind of track
we're working with.
We can work with mono, stereo, surround information,
or any of our B-format first order or second order ambisonics.
It's important for us to look at the parameters of our positioning.
The azimuth is our horizontal plane--
in other words, left and right 360 degrees around us.
The elevation is the up and down access point
that gives us height information in relation
to the picture that we're seeing.
Our distance parameter is the proximity of an object
to the viewer in the 360 space.
With all of these positioning markers in place,
it helps us create the head-tracking parameters.
So as we move around in our 360 environment,
the objects feel like they're either behind us or to the side of us.
Next, we'll look at the Audio 360 Control plugin.
Essentially, this is your master fader playback output controller.
Only one instance of this plugin should be instantiated on every session.
We also have a video player.
This allows us to monitor how our mix is working with whatever visuals
we're working with.
Previously, when we were discussing converting A-format to B-format,
we mentioned the fact that sometimes you want
to rotate your ambisonic field to match that of picture.
Therefore, in our Converter plugin within the suite,
it has the ability for us to make that rotation happen after the fact,
as well as convert from one format of ambisonics to another.
Next, we should look at how to meter ambisonics.
We have two versions of a metering plugin in the suite--
a 360 meter, which gives us average levels as we look at our 360
environment, as well as a meter that actually doesn't
have any metering on it called the Stereo Loudness meter,
which links to our 360 meter plugin.
This allows us to simultaneously monitor head-locked audio
and ambisonic spatialized audio.
And finally, we have our Audio 360 Encoder.
This allows me to take my final mix output and conform it to a video.
[MUSIC PLAYING]
So now I'm going to take you through how to set up my session.
I decided to make my A to B-format conversion in real time.
So here you see four channels-- channels 1
through 4 of my A-format Ambeo mic being routed to the Ambeo Mic master
track, on which I'm putting my Ambeo A to B converter, which
is a free download when you use the Sennheiser Ambeo mic.
This allows me to import the A-format audio and, in real time,
see the B-format output conversion.
If I play this, you'll see that, on the left-hand side,
I have A-format input, which is my four discrete channels from the microphone
capsules.
I have the ability to rotate that microphone in relation to a camera.
I have the ability to tell it what position it's going to be in.
In this case, this microphone was upright.
And I can tell it output format.
I will see slight variations in the output levels as I click through these.
And this would be an audible difference.
In this case, I'm going to choose to stick with AmbiX as my B-format output.
So right now, I'm converting A-format audio to B-format first order AmbiX.
Right below that, I'm going to instantiate my Spatialiser plugin.
Because it's an ambisonic recording, I'm going to select my B-format first order
AmbiX as the input to this plugin.
Next, I'm going to want to make sure that I
have a video linked to my session.
When I click the Load button, I automatically launch the video player.
This allows me to select the video I'm going to be working with.
It's important to note that you might need to convert from the original file
your camera captured into a format that is more acceptable by the DAW.
Next, I can tell it how I want to view that video format--
whether I want to have a monoscopic or side by side render.
And lastly, I need to connect it to the DAW
by selecting my laptop as the network connection.
I can open that up, and it should automatically
load my video client as well as my video into my Spatialiser plugin.
Once I do it on one of my Spatialiser plugins,
it will link to all of the tracks that have Spatialiser on them.
Remember that with an ambisonic sound source,
you already have a front, back, left, right, and up and down.
So really, I need to make sure that my positioning is
correct to how the microphone was in relation to the camera.
But beyond that, I don't want to deal with the positioning markers too much.
I'm going to leave that to any mono or stereo sound sources.
Next, I'm going to move to my spot tracks.
For this recording, I had a string quartet at Symphony Hall, each one
with their own individual microphones.
I have violin 1, violin 2, cello, and viola.
Each one of these tracks is going to have their own individual Spatialiser
plugin, each with a left or right mono input.
So to demonstrate spatialization, I'm going to take my first violin
and show you how I lock it to picture and enable
it to be head-trackable information.
Because this is a sound source that I want to be emanating from the violin,
I can either move it on my video, as you see here,
or I can use my 360 panner to move it around.
I can also enable tracking information, which, once the video is playing back,
will become automation data.
That means that if this player were to get up and run around the room,
as I turn, I would hear that following.
Because the sound source is pretty small,
I'm going to want to make sure to get really localized on what
portion of the violin I want this detail in the sound source to come from.
I could also deal with the distance of the microphone from my camera.
And once I feel like I'm content with this,
I can now audition what that's going to sound like in real time
by looking at my video client.
[VIOLIN PLAYING]
As I spin around the room, I will be hearing the change of that sound source
from being in front of me to be behind me on the sides or up and down.
Next, I want to make sure that any of the tracks with the Spatialiser plugin
on them--
so in this case, that would be my four spot microphones and my Ambeo master
track--
are all being routed to the Control plugin.
Remember, the Control plugin is my master output.
In this case, I'm going to want to make sure that Decode to Binaural
is selected so that while I listen back, I'm getting
a binaural rendering of my ambisonics.
I want to make sure that that setting is always
on while I'm working on that mix.
But later, once I want to export this, I'm
going to make sure and remember to turn that off so that I
get a full ambisonic output of my mix.
We discussed the fact that we can have sources that are not spatialized,
but rather what we call head-locked audio.
For this session, I had a pair of microphones set up
as a left, right room.
Those are being routed through my head-locked master track.
This has my Audio 360 Stereo Loudness plugin.
What this is doing is tethering to my master track
where I have my 360 mix loudness meter.
This is, in fact, my overall reference meter
for both spatialized content, as well as head-locked content,
and allows me to monitor my overall loudness.
Which means that if I solo my head-locked tracks, which
are my left and right room, I will be able to open up my 360 mix loudness
meter.
And even though on the Stereo Loudness plugin
it doesn't look like it's doing anything,
I can now see that the Stereo Loudness meter is, in fact, sending information
over to my 360 meter.
If I mute those tracks, I should now not see any signal,
because the rest of the tracks in my session are muted.
And if I open up any of my spatialized ambisonic sources,
I now see ambisonic metering on my 360 meter.
In addition to that, I've chosen to put a Converter plugin on this same track.
What this would allow me to do is either convert
from one format ambisonic to another or rotate
the field of my ambisonic microphone to match a camera.
In this case, I don't necessarily need to do that,
because I haven't yet rendered a file with B-format conversion.
So I can make that rotation in real time.
[MUSIC PLAYING]
Once I feel like I've got a mix that is working both dynamically as well
as spatial-wise, I can now be ready to export this mix out of my DAW.
I'm going to want to make a selection of when
to start and stop the final bounce.
Additionally, if you remember, the Control plugin
is doing a real time binaural decoding.
So right now, before I bounce out, I want to turn off Decode to Binaural.
This is going to enable me to output all nine channels of my second order
ambisonic output when I bounce.
Once I have all of that selected, I want to make sure
that I'm outputting in 48 kilohertz.
Even though I can make my recording at a higher sample rate,
it's recommended that the final bounce is at 48k.
And I want to tell it that it's nine tracks.
Now you can see that the Control plugin is outputting all nine
channels in non-real time, and I'm getting a bounce
of my final ambisonic recording.
[MUSIC PLAYING]
Now that I have a final bounce of my mix,
I can use my Encoder plugin to deliver a final video.
At the top of my Encoder plugin, I can select what format
I'm going to encode this video for.
In this case, I'm going to be creating a Facebook 360 video.
I can also tell it what kind of spatial audio
I'm going to tether to this video.
For me, this is going to be a B-format second order AmbiX.
And I'm going to load that audio file.
There we go.
Next, if I had head-locked audio, I could load that file,
and it will merge as part of my final video upload.
And finally, I want to tell it what kind of video I'm going to be creating.
And I can load the original video file that my camera
recorded, which is full resolution.
And I'm ready to encode a final master video.
So now that we've gone through setting up your session,
converting A to B-format ambisonics, mixing your session
and spatializing it, outputting that mix out of your DAW,
and encoding it into a final video, you're
ready to upload that to Facebook, YouTube, or any other viewer format.
I would recommend that before you release it to the public,
you do a private screening of your upload online
to make sure that it's actually working the way you intend
and that your encoded ambisonic information combined
into your video the way you expected.
I hope you have fun exploring the rest of the parameters inside each plugin
and making some creative projects.
[MUSIC PLAYING]