[MUSIC PLAYING]
GUY PAVELO: Hi, my name is Guy Pavelo
with Far Right Productions.
I'm the lead creative and show designer programmer
and alongside Steve Kidd producer
of Psyonix and Rocket League Season Two Wrap
Up Kaskade Event.
This was a two part performance with the first tracks being
within a true studio environment,
but to then merge into Rocket League's Championship Field
and ultimately culminating within Rocket League's Neon
Field World Environment.
I wanted to create new types of XR shooting
and experiences to create new XR perspectives
like over the artist's shoulder, a more broad festival
perspective shot, a blimp camera,
fly by cameras, and helicopter style shots.
We wanted to move out of the XR bubble of restraint
and focus on a much grander scale.
I felt the need for the show to be time-coded
and fully DMX console triggered.
This was for repeatability and the fact
that we predicted requiring many camera
takes of different angles and approaches,
as well as knew we wanted a seamless show experience.
We controlled hundreds of elements
within the Neon Field World, but also controlled and developed
over 800 internal custom DMX fixtures
to extend the show environment.
We investigated a few internal, as well as
external engine tools and elements ultimately working
with all of it now to create custom internal world
control, lighting, and laser fixture type and elements
to attain frame rates needed, as well as to create a larger more
immersive experience and look that I was after.
We were able to progressively push our custom Actors
and tools with a step by step approach pushing their limits,
seeing what was working and faltering,
and constantly creating new fixtures
and adjustments as our team programmed the show.
We ended up in end with a fully time-coded performance,
roughly 110,000 queue triggers, and a common frame
rate within the engine of 80 frames per second.
We were also able to record all universes of data
directly into the Unreal Sequencer for any desired
sequence burns and camera record demos with full real time
playback.
I wanted to use only Rocket League game asset
elements within our show.
For example, we took things like the car boost element
and created it into our own fire elemental pyro.
We created a custom audio trigger system
to relay and mimic the Rocket League Neon Field EQ System
real time to Kaskade's live Performance Audio.
I added in the Rocket League bunny girl as a dancer
within the neon field area.
And we added color, speed, intensity, and opacity
controls to our originally provided asset
from Psyonix. We added in more PA towers
directly from the Neon World Scene.
And we added elements like the balloon dog and rubber duck
having them float like a beachball balloon
at a real outdoor festival.
Ultimately, as a team, we were able to utilize and push
the engine to create a one of a kind type
of performance for Rocket League, Monstercat,
Psyonix, Kaskade, and ultimately the Fan.
The upgraded and clean tools within the engine
were paramount to our success and consistently
proved to be next level.
DANNY FIRPO: Hi, my name is Danny Firpo from All of it Now.
I was the XR screens producer for this project.
And today, we're going to talk a little bit
about how Unreal Engine was used in multiple phases
of production.
So right off the bat in preproduction,
we were working with Guy and the FRP team
on the show control and design elements for this project.
And we had to create separate branches
so that Guy could continue to program
and produce the show while we worked
on some of the creative front end and technical back end.
We were also able to work with the director doing
virtual location scouts before we arrived on set
and allowed us to find shots both in XR
and in our cinematic pipeline.
For the onset XR production, we created both front plate
and back plate branches, which allowed us to put elements
both in front and behind the DJ and really immerse him inside
of the virtual world.
What made this tricky is for those iconic reverse shots
where we were looking over the DJ's shoulder,
we had to flip our front plate and back plate
elements because the camera's essentially
180 degrees from where it normally
is through the rest of the broadcast.
We solved this problem by breaking the project file up
into a series of sublevels, which allowed us to easily
toggle elements between front plate and back plate,
as well as being able to work in parallel
with multiple technical artists working on different elements
of the scene.
Unreal Engine was also used in the post-production phase
for those cinematic intro and outro segments,
as well as other environmental elements
that we injected into the stream throughout the broadcast.
This gave us true creative cohesion
because Unreal Engine was used from start
to finish in this project and created a seamless production
pipeline.
BERTO MORA: Hello, everyone.
My name is Berto Mora and I'm an XR specialist at All of it Now.
Today, I'm going to talk to you about one
of the amazing features that we use
to create a fully immersive environment within the Kaskade
Virtual Concert inside of Fortnight.
We were able to use the latest virtual zoom feature
within the Disguise XR software in order
to go beyond the limitations of physical space.
This feature was previously only possible within the real time
green screen technology and now possible
with LED screen technology.
Here we are sending the Unreal Engine environment
into Disguise.
Within here, we're able to break
the boundaries of physical space and extend
into the virtual world with the virtual zoom.
JEFFREY HEPBURN: Hey, there.
My name is Jeffrey Hepburn.
I'm the lead technical artist at All of it Now.
And in this, we're going to be going over
how we used Unreal Engine with DMX data
to do live virtual production on Kaskade's Rocket League
Llama-Rama event.
So one of the really cool things about having DMX data in Unreal
Engine now is that it kind of bridges
the gap between traditional live show LDs
and real time content creation in Unreal.
Basically anything that can be driven with numbers
can then be driven and programmed as content.
So if you want to drive position,
scales, rotations of objects, particle systems,
have it feed in the value of the emission
rate, the number of particles, triggering on and off--
the sky's the limit on that front.
So for the show we were using the latest version
of Unreal Engine 4.26, which came with some really
big improvements to the DMX implementation
within the engine, which was super
vital for this production.
In Neon Fields alone we had almost 50 custom Blueprint
Actors.
Many of them being Rocket League assets
given to us by Psyonix and then exposed
with parameters for live show control.
This was being driven by about 7,600 channels of DMX data
across nearly 800 individual patched fixtures, which
is a ton.
But we managed to get all of these running live
in real time for the shoot.
The DMX testing and debugging tools
within the engine we're also greatly improved,
which made my life as a technical artist much easier
because I could then evaluate the performance
and functionality of the Blueprint Actors
I was making to ensure that they would
work as intended before handing off to the FRP team, which
made the whole collaboration process that much easier.
So the biggest difference between a virtual production
and a traditional in-person live show would just be performance.
For a traditional live show having dozens,
hundreds, even thousands of lights on stage
really isn't a problem because each of those fixtures
is itself its own hardware, which
will take in DMX commands, interpret them, and carry out
those instructions.
For a virtual production this isn't the case.
And so you going to have to be a little bit more
clever about your content design in order to achieve real time
performance because now you're having
to think that all of this DMX data
is being sent to a single server, a single CPU.
And so that has a lot more performance implications
that you have to consider like a game dev more than an LD,
where in that case, your biggest limiting factor
is just going to be how much data your console can output.
But at the end of the day, clever design
is going to help you achieve your content
goals in a performant manner for real time.
As an example of how we achieve this in Kaskade,
I'm going to use this marquee trim.
Now, we wanted to treat this like a traditional marquee
with hundreds of little bulbs.
Ultimately, that was going to be too performance impactful.
And so our solution was to actually treat this more like
a neon tube with gobos. We had four different gobo
layers each being sent just a couple of DMX parameters.
On the material side we had the different layers
with different patterns of gobo shapes.
And then the final output is a gobo
where we have RGB control for the colors and adjustments
on that side and then different opacity masks for the gobos
and velocities to control and layer
how we want this dynamic gobo to be interpreted for a real time
fixture.
So the big takeaway here is where possible,
if you can take what would be hundreds
or thousands of fixtures in a traditional show
and combine them down into a single fixture
with some clever shader design,
you can still achieve your content
goals but in a performant manner for real time.
[MUSIC PLAYING]