Narrator: This belt is getting destroyed in real life.
Then it heads to a photographer to get scanned.
All so it can appear convincingly worn and weathered
in "Call of Duty: Modern Warfare II" ...
Ghost: Enemy KIA.
Narrator: Which has been called
the most realistic-looking video game ever made.
The team behind the popular franchise
has created and distressed
tens of thousands of real props,
set pieces, and costumes like this.
Direct. Target destroyed.
Narrator: But they didn't stop with objects.
They also stepped up their performance-capture tech
to create an insanely true-to-life feel and experience.
We took a look inside the game's LA studio
to find out how they transport our world
into theirs.
And it all starts with a process called photogrammetry,
or capturing high-res photos of a person or object
from every possible angle,
then stitching these photos together
to form an incredibly detailed 3D replica.
It fires off all five cameras at once.
The turntable will turn 8 degrees,
and then it takes another shot.
Because we're capturing it from down below and up above,
we get every angle of this part,
which allows our photogrammetry software
to reconstruct this as a 3D model.
Narrator: The "Modern Warfare" team can then manipulate
this photorealistic 3D model in the computer.
Photogrammetry makes it easier and quicker
to enrich the world of the game
with new objects and characters.
Traditionally, video-game characters are built digitally,
and that's exactly how "Call of Duty" worked until 2015,
when the team began incorporating photogrammetry.
Now the team can build a character in one week
instead of six.
Bernardo: And the output is a mesh,
very, very dense, with a lot of details.
That can go to our character artists,
and using this data, they're going to build a game asset.
Narrator: One of the most time-saving parts of this process
is the ability to scan costumes
in all of their glorious details.
See, when building costumes in the computer,
a good chunk of the time is spent painting in wear and tear.
"Call of Duty" is a war game, after all;
nothing about the costumes should look pristine.
With photogrammetry, instead of digitally simulating
each scuff, chip, and stain,
you can create all of those details in the physical world
and scan them into the game.
So, this is our distressing station.
This is where we take gear
that we're putting on characters in the game.
We distress it to look like it's been out in the field
for an extended period of time.
'Cause when you get something brand new,
it really wouldn't scan well.
It doesn't have any sense of realism.
Narrator: They use rotary tools to rip and fray the costumes
and sanders to make the fabric less starchy and rigid.
Chris: The fabric tends to feel older if you sand it down.
Narrator: And to make the clothes feel
like they've been out in the desert
for weeks at a time ...
Chris: We have bags of fake dirt.
You can actually just kind of plop it down
right on the item.
Narrator: This process only looks haphazard.
Story matters.
We try to put it where we think, if worn in the field,
where would a soldier kind of lay on the ground?
Where would the most scratches occur?
Where would the most holes happen?
And where would the most dirt accumulate?
When we distress stuff, we try to kind of tell the story.
Narrator: That story might involve stains or bullet wounds.
Chris: We used to use things like coffee and tea
to stain things, but those ended up
creating a bad smell over time.
So now we use things like black ink mixed with water.
We have fake Hollywood blood here, so if anything
needs to look like somebody's been injured in it,
we have that.
Narrator: Putting a fully costumed actor
into the photogrammetry dome
helps the team capture other subtleties too,
like how fabric hangs and sways on the body.
They also use this approach on props.
So, this is Ghost's desert mask.
You may recognize it
from the opening of "Modern Warfare II."
It was actually created practically,
and then we scanned it here in this volume
and a 3D artist took over and built it for the game.
Narrator: Many props get scanned into the game
through an alternative approach called an Arago rig.
Valerio: It's a new system that we implemented recently.
It's just for prop scanning mostly.
It's a step up from having a bunch of cameras.
This system allows you to have a single camera on.
The miniature is on a rotating turntable,
and the machine will just move the camera around for you.
So instead of figuring out where you need to be
in relation to your subject,
you just do that on the software.
So you can prearrange all your camera needs,
all your camera positions,
and input all your camera settings,
and then it will just do the sequence for you.
This is actually packable and transportable.
You can take this to a prop house
and then scan hundreds of props a day.
Narrator: Here Valerio is using the Arago rig
to scan in one of the game's many miniatures,
handcrafted models of set pieces, weapons, and vehicles.
Once scanned in, these models serve
as the emblems for Battle Pass,
the system that rewards users for playing the game.
And they aren't designed to feel as strictly realistic
as other parts of "Modern Warfare."
Overall, we're shooting for hyperrealism.
We want the game to be 100% realistic.
But this is not necessarily in-game content.
This is displayed outside of the game.
Narrator: Because of that distinction,
they want the emblems and rewards
to feel bespoke, handmade, and collectible,
qualities that handcrafted miniatures lend themselves to.
Valerio: Then it's easy to just tell your artist,
"I want this. This is my vision. Just go build it."
Rather than try to find it in the real world.
Sculpting it by hand allows you to retain some of that
natural feel, natural detail, like miniatures.
Narrator: The automated system provides a lot of control
over scanning these individual objects into the game.
But scanning in real-world settings and environments
requires a more free-form approach,
consisting of a handheld rig
with a high-resolution Sony 7R4 camera.
Valerio is demoing this approach on a miniature here.
But normally this is what he'll use out in the field
for real-life settings and set pieces
that can't be placed on a turntable
or stuck inside the dome.
We rely on these high-resolution cameras
to be able to pull detail from the real world.
I would say 80% of the scans that I do
happen out in the real world.
So it could be an urban environment
and we could be shooting a lot of ground,
a lot of streets, street elements, trash cans, curbs,
a lot of small detail that's going to help
really push the fidelity, the quality of the game.
Narrator: And then there are natural textures.
Valerio: We also do foliage, for example.
A lot of rocks. Sandstone, granite.
I did a lot of sand up in Northern California.
Coastal scans.
Narrator: For the Mexico setting of "Modern Warfare II,"
the team spent a week doing photogrammetry
in the Mexican state of Chihuahua,
including aerial photogrammetry with drones
to capture expansive landscapes,
with the goal of upping the level
of natural detail in the game.
The team dug really deep into, for example,
researching in the real-world locations
of what is the light setting
at that particular place in the world, right?
So if we're taking a map and we say to ourselves, like,
it's geographically about here in the world,
sending out teams to actually measure, like,
at this time of day,
this is the luminosity of the sun,
this is how the light is bouncing around, et cetera.
Narrator: But photogrammetry has its limitations,
especially when it comes to light-reflecting surfaces.
Valerio: Anything that's reflective,
even, say, glassware, cannot be scanned.
Narrator: That creates challenges
for the many vehicles in the game.
Another limitation of photogrammetry ...
Valerio: It only works on static objects,
like rocks, curbs, trash cans.
Anything that moves,
like a tree canopy or even a person's hair,
usually cannot be scanned really well.
So water is something that cannot be scanned.
Even when we scan characters and actors,
we don't really get the hair
to work too well with this technology.
Narrator: That's where they use computer simulations
to govern the physical behavior
of moving elements like hair or water,
helping create the Amsterdam canal seen here.
But Amsterdam would be a ghost town
without lifelike characters to populate it.
And to create dynamic characters,
you can't just stick actors
inside the photogrammetry dome.
As many facial expressions as actors do in there,
it's still a static process.
Photogrammetry is one of many
reality-capture techniques or methods.
Narrator: The other method the game relies on
for its characters: performance capture.
This process differs slightly from motion capture,
as it records not just the motion of the body,
but also the face, fingers, and voice.
We capture the performance of the body,
we have emoti-cameras to capture
all the subtle performance of the face,
and we also capture audio.
So everything is captured all together.
It's like basically shooting a movie at this stage.
Narrator: But it's also different
from filming a movie in some key ways.
For one, the actors' expressions are captured
via facial cameras mounted to their heads.
Because some detail gets lost
in this facial-capture process,
performing for video games
is a bit like performing onstage,
where actors rely on
slightly exaggerated movements and expressions.
And unlike movies, video games are interactive,
and players can fall, run, climb, and fight indefinitely.
That means actors must record hundreds
of exertion sounds and other noises,
sometimes doing dozens of takes on a grunt
or sigh
to keep their characters from sounding stale or robotic
over hours of gameplay.
The finishing touches come in the sound design,
where the team aims to transport
as much content as possible from the real world,
recording out in the field and with real weaponry and props.
Steve: Every time you shoot a gun in the game
and you hear the brass eject and land on the ground
on the different surfaces,
whether it be wood or concrete or the metal surfaces,
we record all of that.
We do absolutely every bit of recording we can,
particularly when it comes to weapons and vehicles.
On this last game, we recorded all of our weapons, tanks,
real practice rounds from an RPG,
three different helicopters.
Anything we can to make the game as authentic as possible,
we're out there trying to capture sound for it.
Narrator: It's all part of raising the stakes
and keeping them high
to fully immerse players in the game
and make their actions feel like they carry weight
when simulating something as perilous as war.