[MUSIC PLAYING]
DOMINIC AUDET: So Moment Factory, first of all,
we're a creative multi-media studio from Montreal.
We specialize in live entertainment.
We're bringing people together.
This is why we say "We do it in public."
Now, after 20 years, we have many different areas
of activities.
So we do theme parks, airports, public spaces, museums.
But always, it's the "We do it in public" motto.
That is the common thread into everything we do.
So we do not do home entertainment or one human
with the screen.
But we try to use all this innovation and technologies,
like what Epic is famous for right
now, to bring to this live, location-based entertainment,
where people connect together.
So we kind of use the same technologies
but in a different way, to our perspective.
And we're always going to stay true to this core value
from our group.
[MUSIC PLAYING]
Well, the types of careers and professions at the Moment
Factory vary a lot.
And I would say, one thing particular
is that under the same roof in the studio,
we do our own production for content
production, video production, like VFX,
3D shooting with cameras.
Also interactive, so software development--
we do the frontend and backend--
that we produce ourselves.
Also, we have the Scenography and Technical department,
where we do the specification.
Obviously, we're not building the projectors.
But we do all the design.
So under the same roof, having these core competencies
makes Moment Factory very unique because we
can share these different points of view
from different professions when we do a concept and a design.
[MUSIC PLAYING]
Augmenting reality has always been very important
for Moment Factory.
This is really what we're doing.
But mixed reality for Moment Factory
today is using all the virtual reality, new technologies
in gaming and mixing it with the projection
mapping and real audio-visual effects
in stories that we're bringing.
So it's really the bridge between
the virtual technologies and the real-life technologies
together.
It's what we call mixed reality.
So it's very important.
And we're working on new innovations
about space recognition, people recognition, object
recognition, so that we can really
have everybody be able to interact leveraging all
of this innovation that is happening
around the world, which, like Eopic Games,
is one of the leaders.
And we're working together to bring these new innovation
to the real world and with real people.
[MUSIC PLAYING]
CELINE MORNET: I arrived at Moment Factory eight years ago.
Before that, I was a creative developer.
I used to experiment with web technologies.
In the Interactive team, we have some software developers,
some interactive designers.
And our mission is really to explore the interface
into the physical world.
[MUSIC PLAYING]
Our first immersive project or immersive performance
was for the City of Montreal.
And it was a 4-minute live performance
with real-time content.
We had the mesh in the front and LED backdrop.
And the performer can interact in real time with this content.
I think it was our first step into the mixed reality.
It was in 2015.
Our challenges are that we have really complex canvas
with LED screens, LED sounds, immersive projection,
mega-high-resolution content.
And it's also a multidisciplinary team
at Moment Factory because we have
a lot of people coming from different backgrounds who are
used to using different tools.
So we started to use game engines
because it's a way to group content artist with software
developer so they can share competencies and assets, even
3D models.
And even technical teams show interest into this tool.
Platforms such as Unreal Engine change
the way we plan our project.
Because with previs tools, we are
able to build a full simulation environment earlier
in the process.
Like in design phase, we can visualize all the features
of our project or in real time pre-render content
in the same environment.
Using real-time tools for augmented live performances
is really to save some render time.
You don't need to come by and wait hours
before you see the results.
You can sit down with the director
and just make your adjustments live.
One of the projects we did with real-time tools
is Kento augmented artist.
It's a Japanese artist who wanted to be the first world
augmented artist.
And it was an innovation project.
And we did a lot of prototyping workshop with him.
We have to bring him with us in all the process.
It was a really innovative project
in augmented performance because we
had to track his 3D Skeleton with the camera tracking
system.
We had to have this silhouette with the depth camera
to have the occlusion and to be able to recreate
an environment around him.
He wanted to be able to go inside and outside
and to have a portable augmented broadcast system, so even
more constrained than when we are
doing traditional augmented broadcasts.
We did the other augmented performances
with high-speed tracking system to allow the performer
to interact in real time with the content and with the space.
[MUSIC PLAYING]
The challenge is to do a live broadcast show
with Billie Eilish, where, first of all,
short timeline of production.
Then the performance of the builds,
we had to sit our content artists who
are doing some high-quality content with our developers
to help them to optimize the build.
Because when you have some hi-res surfaces,
you have to be frame-perfect.
So you really have to have good performances
and to synchronize all the cameras with the content.
[MUSIC PLAYING]
It's also to bring the real-time way of thinking to the director
and to the creative director because you have
to think your show differently.
With augmented broadcast and real-time engine,
the line between physical world and virtual world
is really blending in the same direction.
DANIEL JEAN: There's been a handful of projects
that knocked on our door from collaborators
who had crazy ideas or really wanted
to push the boundaries of innovation
and needed some sort of help to make it happen.
When I think about Nine Inch Nails in 2008,
"Lights in the Sky"--
how to control all these devices on stage with movement, sound.
When I think about Madonna's Super Bowl--
creating this mapping on the field with 3D optical illusion
effects.
Going to Red Hot Chili Peppers--
1,000 of these winch tubes going up and down.
And we just created some sort of a patch
to create the shapes that would move the tubes around.
But we'll also display the lights on the tube.
Muse Drones tour was a very ambitious stage with drones
flying all over the place.
And there was this challenge of mapping real time
on the drones.
But also, we had interactivity on the stage
to follow the band members and create effects around them.
And then the pandemic hit.
So everything that was happening in real life
got transferred to a screen.
And one of the first things we did
is take all the knowledge and all the workshops, the labs
that we've done in the past for artists,
and we built this augmented life tool.
So part of this lab was, how can we
enhance the performance of everyone
that's doing from their living room,
their kitchen, their backyard?
So we developed remote tools to enhance the performance.
I think all this technology really helps
to enhance the performance.
But really important for us was how
are we going to connect the artist with his fans,
and how we're going to connect the fans to the artist.
So we use a multitude of different platforms,
perhaps like Twitch, in order to have
the fans to interact with the performance
and have the performer to interact with the fans.
And then came our collaboration with Unreal
to develop the new feature, the 4.26 DMX.
So basically, the idea was to bring
our expertise, the live show design
in the physical environment, and bring it
into the digital environment into Unreal--
so stage design, special effects, lighting,
everything controllable within the Unreal platform.
[MUSIC PLAYING]
One of the great opportunities that we had last year
was Billie's XR performance.
So following the tour, which only happened for three shows,
in Florida in March, we went back to the drawing board,
looked at all the content we created for the show
and saw how we can adapt it for the 3D
environment in XR studios.
Most of the content was built for screen.
So we had to see how are we going
to break down all the content and have the layers to fit
in this 3D environment, so that when we played it on the XR
stage, you can see the foreground
elements, the background element,
and the extended elements.
And for us, who have been designing
for more than a decade concerts in theaters or arenas,
we always think about the 3D space and how are you
going to live this experience, how
are you going to share it with your friends,
how are you going to connect with the performance.
And now this layer of having a feed for people at home
is a challenge.
It's going to be a challenge.
And we're going to have to think about
how do we bring the emotion to them, how do we connect them
with the performance, how we connect
the performance to them, how we connect them
amongst each other.
So it's all questions that we're having right now.
The future of entertainment, I would say,
I could start with what's happening right now
with the pandemic, with everybody stuck
behind their laptops, doing their human life
through the screen.
I think it expedited what was already
happening in the entertainment industry,
but maybe put in one year what should have come in five years.
Which is this blurred line between what's virtual
and what's real for humanity.
So all these new technologies for the entertainment
are bringing us with--
I think the gaming industry is leading the pack right now
for innovation.
So more and more we're going to have
a more gamified life, which is enhanced with virtual reality.
And we won't see the difference between what's real
and what's not real.
[MUSIC PLAYING]