NIKO: So Snoop Dogg has asked me to bring Tupac back to life for a music video.
So long story short,
Jesse Wellens, who's a friend of ours, and Snoop Dogg
are working on a new music video and they have an idea
and that's to take a classic clip
from the video music awards from the 90s, from MTV, where Snoop Dogg and Tupac are sitting side by side
and they want me to take that clip,
and make Tupac
lip-sync to the new song that Snoop is making.
So, not ONLY do I have to impress one of the most iconic
rappers of all time who was close personal friends with Tupac,
I also need to stand up to the awesome Coachella
hologram that they did.
But you know the biggest burden of all?
The highest pressure that is sitting upon my shoulders here?
Is I need to impress Jake.
Because Jake loves Tupac.
[Corridor Intro SFX]
[MUSIC]
JAKE: Growing up, as a white suburban kid from Minnesota about 12 years old,
the first time I heard Tupac I was like, "WHOA!"
[MUSIC: 2Pac- Hail Mary]
"Holy... Wow!"
[ALL LAUGHING]
JAKE: You can ask anyone I knew back then,
but I was a...
Very, very, big Tupac fan growing up back in the day
I would even go on like "Is Tupac dead?" forums, did all the research on the conspiracy
whether he was still alive and obviously through that I obviously listened to a ton of
Snoop's early stuff and actually honestly Pac's music back then
actually turned me on to like, real world issues,
um,
and made me think a little bit deeper about everything.
That's the real contribution that he made. To me anyway, and I think to many other people
JESSE: We're just like, "what's just something that we haven't seen?"
and we're like "we haven't seen...
Tupac deep fake" and I was like you guys are the ones to do it and it's like, "let's deep fake Pac"
In a Snoop Dogg music video.
Could break the internet, could do something on the internet,
with that little piece alone.
SNOOP: Morning, dog.
JESSE: Lemme introduce you guys to- to Dogg,
this is Corridor Digital,
they're the guys that're gonna be doing the deepfake
for the Pac shot.
NIKO: Hey, nice to meet you.
JESSE: You're sitting at the table, yeah.
JESSE: So, I just wanted you to meet 'em, they're hooking us up
JESSE: All right, let's do it, we're gonna get- Imma come back.
[LO-FI CHILL HIP-HOP ANIME BEATS TO STUDY/RELAX TO]
JESSE: I was thinking we could do it again
If you have like a backwood and some really good-
like some green-ass weed or something, some really good weed?
I mean like, yeah.
SNOOP: Something that'll look good for the camera
JESSE: Look good for the camera, 'cause then we could like
basically get it right through it pull out and then you're like twisting it.
SNOOP: Gotchu, lemme go get it.
JESSE: Yeah, that's fucking great.
JESSE [VO]: What I learned from working with Snoop is- he's super humble
really nice guy, easy to work with as long as he's like, into the project,
and his time is like, being used effectively.
Don't smoke pot on set like that's not good. It's hard, as much as you want to,
he's got weed everywhere and you're like [STRAINED NOISES]
'cause the last music video I learned, the whole crew thought it was okay to smoke weed,
'cause he was smokin' weed, so...
That video, the last shot was just like a MESS.
And everyone was like "Duuuhh, like, I don't know what to do..."
And we're just like "Oh my god…"
So don't smoke pot,
JAKE: Tupac and what you did with him, there's such a legacy left in the music world
based upon what you guys did then, and now.
And I just- I wanted to ask you personally like what did you learn from a figure like that,
being friends with him, and what can people take away
into their own lives from what he did and how he lived?
SNOOP: I think what I learned from Tupac, was basically a different kind of work ethic
I always had a good work ethic about myself as far as being timely and being on point
and being a professional, but
he just showed me how to be a little bit more faster at it as far as like getting to the
to the meat of it all. Not really just listening to it all the time, and not
absorbing it but more that's just doing it, and keep doing it and doing it and then let the engineers mix it and master it
and not fall in love with it, but fall in love with the craft of being able to do it continue to do it.
So, I felt like that was something that
was passed on to me and now I'm showing it through my work ethic as far as the different things that I do, and
the way I diversified myself, and it passes that on to the young generation and show them that
you can do the same thing that I'm doing 'cause that's an old man if I'm doing it.
He was a young man. You got three steps to go ahead and beat me out,
so go ahead and get to it and do it.
JAKE: 'preciate that.
SNOOP: Thank you. My man. 100.
[LAUGHING]
NIKO [VO]: Aaaaand now I have to do a deep fake of Tupac.
Or, Schnoop Dogg? [LAUGHS]
NICK [OFF CAMERA]: Schnoop Dogg! [LAUGHS]
NIKO: I need to still pull this off
I still need to bring Tupac back to life and not just in a blurry deep fake,
but like it needs to look good.
So here's my plan: I'm going to use the old model that I used for our Keanu Reeves video
I'm gonna be using that model from Deep Fakes Lab here,
because I already have a ton of time put into it, I have like a week of
GPU time put into it on just training random faces.
Basically, I can take that and hopefully, quickly turn around in a couple days this deep fake of Tupac.
Alright, so here's the clip. It's hard to find high-quality versions of this, which is challenge number one, right?
Trying to find high-quality video from 1996.
So for me to do a good job of this, I need to rely on what we know.
We pioneered some things with our Tom Cruise deep fake that no one had done with deep fakes before.
We had an impersonator come in
He had the mannerisms, he had the voice.
in this case I don't have to worry about the voice, but I need to worry about everything else.
So, we have a friend, his name is Josh Haraway, and he is a professional Tupac impersonator. Check it out.
This is not Tupac. This is Josh.
[LAUGHS]
Looking a LOT like Tupac. So anyway, once he like, takes all the stuff off,
he doesn't look like Tupac at all.
But when he's in costume and playing the character, it's spot-on.
Y'know, it's the mannerisms, it's the attitude
It's how you kinda carry yourself on camera.
JOSH: WEST SIDE!
JOSH [VO]: I worked on the Tupac, uh
2012 Coachella hologram with Dr. Dre,
I was uh, like, that- that was a fun time.
NIKO: And it's pretty interesting
technology because the computer's learning how to encode a face just like how a
JPEG is how you encode a picture or an MP3 is how you encode an audio file.
You basically have a face encoder and a face decoder and you just running one face
dataset through a different face decoder, so it basically paints that face with the features of a different face, so…
JOSH: I see a government contract in your future [NIKO LAUGHS]
JOSH: So yeah, I think we can do this for sure man
NIKO: I'm going to need to record Josh in full costume
Matching this costume, in a lighting setup that matches this lighting setup, on a green-screen
I'm going to need to erase Tupac from the shot and replace him with Josh,
and Josh is going to be giving us the performance,
lip-syncing to the music, and then I need to deep fake Tupac's face onto Josh.
We're actually super super lucky, because A) the camera is locked off,
and B) Snoop is probably very very high, and sitting still and not moving.
I can isolate Tupac really easily here
I mean, there's a little bit of shadow and stuff on his arm, Snoop is zoned hard right there
Clint on a scale of one to ten and how high do you think he is in this picture?
CLINT: Probably like a… six. [SAM LAUGHS]
WREN: Dude, I don't think he's ever smoked in his life. [NIKO LAUGHS]
CLINT: Niko, when do you think it'll be done?
NIKO: I have one week to do it. You have to work extra fast when you're
working for Snoop Dogg, 'cause he measures everything in dog years.
JOSH [AS TUPAC]: Yo, this is Tupac Shakur,
and if you want to see more interviews with some of the greatest musicians of all time
like myself, subscribe to Corridor Crew!
Twister? Twitters? Can somebody explain to me what "TikTok" is?
NIKO [VO]: So I need to train the encoding and decoding…
part of this, to make it work
and the more time I spend training, the better it looks. But to do any of the training,
AT ALL, I need pictures. I need pictures of these faces
I need lots and lots of pictures Tupac in every single lighting situation,
every single expression, every single angle.
You know what the biggest issue is? everything filmed in the 90s has interlacing look at that interlacing.
Look at that interlacing!
There's a Captain Disilusion video on interlacing if you want to learn more about it,
but your image files in your video have these little lines on everything if there's motion,
which you can see right here.
I found three videos that were high-quality enough that might work.
I have this interview here- decently high quality, not interlaced,
this interview, which is the best quality I found,
and then finally we have this music video,
which has interlacing, but I just have to go through and manually cut out all the interlaced frames.
But, I have another gigantic problem:
everybody for some reason decided to film Tupac from the same side
they have to film from the nose-ring side! So I don't have data
for the other side of his face,
so I have to go into Premiere After Effects, flip all my footage, and re-export it again
so now, all these same shots just flipped.
Alright, so yesterday we had Josh come in we film with him and it went great, great performer
and Jake did a great job being a costume designer
we matched the lighting pretty well, I think, green screen looks good.
So I've gone through them extracted all of his faces, so these are all of his poses here.
And with those, I was able to run the deep fake training.
It's gone for 24 hours now. So I'm encoding with Josh's face and then taking the data and decoding with Tupac's face
Aaaaand here's what I got
It's getting there, it's getting there. It's blurry, but you know what the biggest thing is?
It's flickering.
It's flickering all over the place. As he moves, it's going bright and dark and like changing weirdly
The reason that's happening is because I only have like three sources for Tupac
and the lighting the situation's almost the same between them.
It's only seeing his face under like, three lighting conditions.
It doesn't know how to draw the face when the lighting situation is different
Here's the extra funny thing. Look at his teeth. He has one central tooth!
[NICK LAUGHS]
NIKO: He has both teeth on each side!
His face is perfectly symmetrical!
[LAUGHS] His eyes are perfectly symmetrical,
his eyebrows are perfectly symmetrical!
It's 'cause I only had… WREN: DUDE, TOOTHPAC?!
CLINT: Oh god…
WREN: Now that you pointed it out, I can't see anything other than the fact
HE'S GOT ONE MIDDLE TOOTH!
NIKO: You can't really tell. If I spend more time training it it will get a little bit better.
The biggest issue right now is his skin is flickering.
It's actually pretty easy to deal with, all you need to do is enable a mode in the deep fake training
that randomly shifts the colors of the image
'cause then you get some of the images that are bright, some are dark some are tinted orange,
some are tinted green, you know, whatever. It gives you a little bit of everything.
So the program learns how to deal with the face under any kind of color condition,
and therefore it can give you a non-flickering result.
So I just need to go in and enable that and train it for probably another 24 hours and I'll probably be fine.
WREN: Ridiculous! I am currently working on trying to figure out how to attach a NERF gun
to a racing drone and have it fire remotely. I think it's gonna be amazing.
It's a very very challenging build, documenting the whole thing. Stay tuned.
NIKO: Sorry, not to cut you off. I have a problem… ummm…
NICK [OFF CAMERA]: Oh joy!
NIKO: It's kinda frustrating, NICK [OFF CAMERA] I thought you got some variety in there?
NIKO: Yeah, I thought I had variety in there too, but apparently "random color transfer" just means
"tint everything green and only green"
[FRUSTRATED LAUGH]
Here, so on the left here,
this is the picture that's being fed into the encoding
and it's supposed to be randomly tinted, and now it's just green.
IT'S SUPPOSED TO BE MORE THAN ONE COLOR! Random means other colors! Like, I don't know, pick one…
blue! [BOTH LAUGH]
I spent a good chunk of time trying to fix this. It's like I've reinstalled the program,
I changed all my graphics drivers,
I tried swapping to different models. and it's just… Stuck. Being. Green.
[BARF NOISE] It doesn't look very gooood…
Well, the flickering's gone…
[LAUGHS]
Even though the program's not working, I can do this manually.
I'm going to set up an After Effects composition,
I'm going to take all the frames, both mirrored and the un-mirrored,
and I'm going to apply a wiggle operator to a color balance filter.
so every single value in the color balance is going to be wiggled per-frame.
Wiggle is an After Effects way of
changing a value with noise
basically
Every 30th of a second is going to change the amount randomly,
80 values in either direction from where the value is.
Once for each frame it's going to have a completely new set of values.
And I've copied and pasted this and do every single color balance entry here,
so if I preview this, it's just like a party going on
[LAUGHS]
All right, so I'm going to
manually export new pictures with wiggled color and wiggled brightness,
and then I'm going to use those to train this, and that should theoretically fix the issue.
Snoop Dogg, I don't think he gets angry. I think if he were to see this, and it wasn't very good,
I feel like he'd sit in the corner,
and he'd get kind of quiet, and feel kind of sad, and then his producer friend would be like
"Hey Snoop, you want to record this track?" and he's like
"Not feeling it today"
and I don't want that to happen.
NIKO: All right, so I'm finished. Like I'm done.
No one's actually seen it yet. NICK [OFF CAMERA]: Like, done with the whole shot?
NIKO: I'm done with the whole shot.
I've made it look like it was filmed in the 90s, I've made it look like it was Tupac
I'm ready to show it to people. I hope they like it. Hey Jake. Do you want to see Tupac?
NICK [OFF CAMERA]: The resurrection?
NIKO: Does anybody else want to see the resurrection?
Here we go, no one's seen this yet, so I hope you like it…
SAM: You're bringing Tupac back to life.
NIKO: Here we go!
[MUSIC: Snoop Dogg- I C Your Bullshit]
NIKO: There it is.
[CONGRATULATIONS]
NIKO: He's- he raps once more.
CHRISTIAN: Tupac's rappin' in two thousand twenty.
SAM: That's crazy. NIKO: Is it weird, bringing somebody back from the dead with the deepfake?
WES: Has it ever been done?
JAN: They've already made him into a hologram
[ALL AGREE]
JAKE: This is an honest- this is an honest rendering
WES: Yes, it was rendered with love.
JAKE: Yeah, I guess the next step is to show Jesse and Snoop, then see- see what Snoop thinks.
JESSE: I'm here with The Dogg,
SNOOP: Mmm-hmm.
JESSE: We 'bout to watch- he's about to see if the first time you ain't seen nothing yet, alright.
SNOOP: I ain't seen nothing but I seen a dinosaur. Let's go! All reactions, all real.
[MUSIC: Snoop Dogg- I C Your Bullshit]
SNOOP: Oh! That's hard.
[MUSIC: Snoop Dogg- I C Your Bullshit]
SNOOP: So cold. That's SO cold.
That's hard as fuck, Jess.
That's- that's too much! JESSE: I know you gotta watch it like four times to get it all!
SNOOP: Motherfucker's extraordinary, extra ordinary.
Very very quite contrary that motherfucker is going to be legendary.
That's hard. JESSE: That Tupac's shit's crazy, right?
SNOOP: That's hard.
All- all that shit was hard. I was likin' ALL of that shit.
JESSE: Sweet.
SNOOP: That shit was RIGHT. Like the right… spirit, the right everything on it, the right EVERYTHING on it.
JESSE: I guess he likes it? I hope he likes it! SNOOP: I don't like it, I LOVE it!
[JESSE LAUGHS]
CHRISTIAN: The final shot is delivered, and the music video is coming out any day now,
so head on over to Snoop's channel to watch it,
um, it might even be out right now
so we'll put the link in the description when the video is live.
And make sure you subscribe for more deepfake
experiments on the Corridor Crew Channel.