One volume.
16 Unreal Engines render four cameras, real time.
Seven Unreal Engines,
on top of that, just doing AR.
That has never been done before.
You mean, nobody's doing this?
Anywhere?
We at Fox Sports,
we like to differentiate ourselves by being bold, taking risks.
“Same game, new attitude” was the underlying ethos of Fox Sports and has been for 28 years.
That's kind of been the core DNA of the division.
We are one of the largest broadcasters of some of the major sports leagues
in the United States, from World Series to Super Bowls to World Cup.
We try to differentiate ourselves as, you know, being storytellers,
but also trying to provide a viewer a different way to see things.
We came back from an NAB five or six years ago
where there was a demonstration of Unreal Engine mirrored with other technology.
And it was the first time that I've seen
where a virtual set was actually possible.
We came back from that and convinced our boss
that we had this tight ecosystem in
Charlotte, North Carolina, wrapped around NASCAR, that we had a perfect opportunity
to experiment with this technology and see if we could do something with it.
Always thinking down the road, we would use it on a bigger show
with a bigger audience,
which would be our primary product, which is “Fox NFL Sunday”.
All through the design process for this stage,
it was always going to be green screen.
We were basically going to recreate
what we had in Charlotte because we knew that workflow.
But there had been a lot of noise made around LED and XR,
and so we saw a couple of early demos and as soon as we saw it, it was,
we need to be there.
And if we're not,
by the time we install the green screen, it's going to be obsolete.
We had a lot to deal with.
We had multiple vendors we were working with that
had done components of what we're doing, but not at the scale we're doing
and not the level we're trying to do for real time.
In a live sports setting, it's very difficult to plan.
Whatever happened
the play before, dictates what happens next.
My first Sunday in a control room,
I thought it was — insane way to work.
There are multiple people doing multiple things, talking
to multiple people at the same time,
and if you walk into it cold, you cannot make sense of it.
I guarantee it.
One of the big gains of having a virtual
stage is being able to change the entire world that you're in.
In a live broadcast, you don't have much time to just stop down and reset.
We've got maybe two or 3 minutes when they're running a tape.
The biggest switch from green screen to LED is the live component.
The director needs to actually see the shot they're about to take.
They need to have some form of a preview.
And if one camera is rendering
a specific angle onto the LED, well then another camera is not.
I've never seen anybody do multiple cameras in a virtual stage.
It's always been one camera that tracks and you do the manipulation there.
So that's where this ghost frame technology
was sort of the secret sauce to making a live broadcast happen.
The LEDs that are behind me are running at 240 hertz refresh rate.
And what ghost frame allows you to do is subdivide
those into 60 hertz increments.
That allows us each camera to be split and take every fourth frame.
While the images on the wall can be blurry to the talent,
they can still sort of make it out, which is actually really helpful to them.
But each individual camera output
has its own perspective with corresponding tracking data.
To drive this entire wall for one camera,
we have four engines times four cameras.
Each of those machines is outputting 4K.
Most people at this point have 4K TVs,
that's your entire screen,
that's just one slice of our volume.
Being able to stay at 60 frames at that resolution is a challenge in itself.
On top of that, there's another seven compositing engines
that are taking in our cameras and layering on AR onto that scene.
And then there are also two other Unreal Engines that we use in the monitors
to kind of display a unique tunnel that shows parallax.
So 25 Unreal Engines on the set.
So as you can imagine, it's quite a bit to manage.
We use Erizos to control both the Unreal project when it loads, because again,
that same project has to load against 16 different computers at the same time.
When you launch a project,
it has to push that out and say play in order to get that going.
But also, we have seven tracked cameras by Stype using the Stype Spider system
and they have provided a plug-in within Unreal
that allows us to map the environments to the LED walls.
While we have three squared off walls and a floor, the plug-in allows
for all of the environments to be seen in such a way that you don't see
seams, colors all match to appear natural.
With the Unreal platform, it's meant to just do that day in and day out
without fail.
I think that's a true compliment to what, you know, Epic can do.
There is definitely that “wow” factor walking into a volume
and you could switch to another environment within a matter of seconds.
Just be somewhere else entirely and it feels real.
You're getting the real lighting.
I think our talent's so excited.
I mean, they see the possibilities.
They see the end result, allowing them to kind of interact with XR and AR.
That's where we want to go.
You know, if you consider the life of this set as ten years as ten years
of innovation, we're uniquely positioned to grow with the industry.
It's pretty clear that Unreal offers us
all of the foundation to build upon for, you know, the future for us.
Because real time is the future.
I don't know what other sports networks think about that, but it's clear to me
real time is going to be the de facto way we produce.