Over the last couple of years one of my goals as a business owner has been to deepen our pool of talent and
Intellectual properties so that if Linus the guy gets hit by a bus
Linus media group the company can survive
But I know that for many of you Linus tech tips just wouldn't be the same without the Linus
which is why we formulated a plan we posted this video on our sister channel short circuit where I
Appeared to be the host, but in actual fact, I had no involvement whatsoever
Here's how my team made it and how Monday comm who generously sponsored this time-consuming
But super fun and very educational journey made the process a whole lot
Easier if you haven't heard of Monday comm it's a Productivity tool to help you manage you and your team's work from start to finish
Most deep fakes. You see online are made by taking an existing video clip and then using software to replace the face and
only the face
Unfortunately, our team's task was a little bit more complicated because we wanted our final video to not only look like me
But also sound like me - his voice is proof that his balls are the only two things he hasn't dropped yet
This school meant that we would have to write a script hire a Linus impersonator to read the script on camera
Record and edit that video and then deep fake it
We used Monday comms recruitment management template with a few tweaks to keep track of everything one small problem
There are no lioness impersonators
Well none that are usable anyway that we could find and even if we did find someone who could do Linus's voice
What are the odds that that guy or gal also looks like Linus and I'm not even talking about facial features, although that helps
No matter how convincing the Face Swap the final video simply wouldn't work if the impersonator was say
Built like a brick or had curly hair
Okay
Then so we go the other way then get someone who looks like me and then deep fake the face and voice, right?
Not quite you see deep fake for voice or high fidelity voice
conversion mostly exists in research papers right now
So we couldn't record like a look-alike steak and then morph that audio clip to sound like lioness
Instead we sent high quality audio and script reference pairs to a company called
Resemble AI who use them to train a neural network to generate the voice from scratch?
I give consent to have my voice cloned by resemble
Meanwhile, James wrote a fresh script for our eventual fake line of sackter to read a keyboard review for the Cleve from truly
Ergonomic the machines first pass was a little rough
just when you think you've reviewed every ergonomic keyboard on the planet a new company with a name like
Truly ergonomic launches something like this the Cleve keyboard
Apparently the world's most comfortable keyboard, but with some patience and a ton of help from resemble, by the way
Shout out to Saqib Mohammed our awesome
we were able to get several options for each sentence arrange our selections in premier along with some room tone a voila and
imperfect but definitely recognizable voice clone
Just when you think you've reviewed every organ ama keyboard on the planet a new company with the name
Like truly ergonomic launches something like this the Cleave keyboard. Apparently the world's most comfortable keyboard
That is a pretty sexy sounding robot if I do say so myself
Now for the visuals
we made a casting call on Vancouver actor's guide looking for someone with my hair type face shape skin tone and
Well stature had boy did some people ever not read the ad
but some people did and we ended up hiring a lovely gentleman named Dylan Tebow LTT store calm
I surrendered my favorite clothes to help him get into character and
Well, the Kovach 19 pandemic had made his hair a little longer than the picture, but we decided to run with it
It's up here. Now. That's way too far. How often I use it
And now instead add to control keys right next to each other. How is that helpful? Hey Dylan
Can you act more unreasonably upset about a small detail? No?
losing so many keys that your productivity suffers and it's priced at 300 US dollars a which cut Dylan
Uh, I don't want you to break any company property, but you think you could drop it
Oh my god armed with the footage from Dylan's shoot
We opted to use deep face lab a free tool that is surprisingly easy to use. You just supply a video of your source
So that's me and your destination. So that's Dylan and then use the various batch files to split the videos into frames
Extract faces from the frames train the AI model and then convert the final video
Let's look at each of those steps in a bit more detail
Most deep fakes are made by taking a famous video that already exists and then swapping a new face onto it
But we're actually making a brand new video. So that gives us a huge advantage
Not only are we able to use a set that Linus a shot on before?
But this one has a giant softbox that's always there that makes matching the lighting between our new video and our existing Linus footage
Really easy, but just in case we also took brightness sweep of Dylan from multiple angles, which helped the AI model become more generalizable
We then grab the footage from Linus's short circuit videos plus a couple LT T's with different lighting and face angles and put them all
on a new timeline in Premiere
It's a good idea to remove any extra b-roll sponsor spots or non Lynas faces at this point to save time later
Next we exported the video and dropped it into our deep fake folder with the correct name
So the batch files can point to it the first batch file splits the video into frames
Which you could have actually done in your editing software
which also means less impression while the next one extracts faces from those frames now for the all-important step of
massaging our data set for both size and quality
Monday comms shareable boards helped us work through all the things we needed. You need at least a couple
Thousand images for your source and your destination, but the thing is you don't want too much
So somewhere between two and ten thousand pictures of each face will do nicely
depending on how much time you have and for
Quality you want to remove any faces that are blurry or of other people or are not actually faces at all
It can be a bit tedious to go through so many images even if they are someone's so handsome
But fortunately deep face labs has some built-in sorting tools to help it go faster with that done
It's finally time to train the model
there are a number of knobs to adjust in the training step to ensure that our hardware is being used to its maximum potential and
A few that have to be tweaked over the course of training so that our faces have detail in the eyes
teeth and skin tone for example
it's basically a matter of turning up the key parameters until you get an out of memory error and then
Backing off to the point where you're confident. It won't crash to get the best possible results
We needed to build a pretty badass deep fake ray. We started with the unlimited budget PC from a previous video
It's equipped with a ninety nine hundred KS at 5.1 gigahertz along with 64 gigabytes of RAM
But since video memory is everything when it comes to defects we upgraded the gaming video cards to an RT X Quadro
8000 with a tear-inducing
48 gigabytes of video memory and while we do have two of these cards we were actually worried that SLI would hurt our performance rather
Than help because deep face lab really isn't optimized for us. I guess rip SLI
Fast forward a couple of days or weeks rather actually and our model previews
We're starting to look pretty detailed. That means it's time to merge
this can be done automatically or
You can use the interactive converter to go through your video frame by frame and tweak settings like super resolution
The blur of the mask edge or the face scale after that. Our settings were used to output a final video and
Just when you think you've reviewed every organ amma keyboard on the planet a new company with the name
Like truly ergonomic launches something like this new cleave keyboard. Apparently the world's most comfortable keyboard
Well, the face swap is good but the lip sync pretty far off the body language doesn't look very Linna
See my hair is curly and the voice sounds
Frankly just kind of tacked on at this point. I thought we were pretty screwed
But fortunately while this video was being made a new version of deep face lab came out that supports whole head swaps
Meaning we no longer needed an actor with similar hair
so I took it upon myself to Don the Costco genes slather dippity-do in my hair to make it as
Small and shiny as possible and talk with exaggerated facial expressions
Ltd store calm and we also use this opportunity to collect some Foley
This is the ambient noises. You'd expect to hear when the host handles packaging
For example, the new model showed promise, but it had its own new issues
Number one see how the hair derps out when he turns his head
That's because our source data set contained images from multiple ltte videos meaning multiple
slightly different gelled hairstyles that
Limits the machine to low detail approximations of my hair to make it crispier
We've removed most of the images from our data set keeping only stills from the original short circuit video
But that didn't end up solving it the hair looks great when I'm facing forward
But since I actually never turned my head in that video we had no data for my right side
So I did some detective work and I learned that line is shot that short circuit video on November 29th, 2019
so I looked through our internal email and chat history to try to find out if you
Shot anything else that same day and as luck would have it
He did including multiple takes or he turns the correct way over and over again
Issue number two. You can still see some remnants of James in there
handsome
But not necessary to fix that we ran the merge three times to create three different
Masks then brought those into premiere and created a composite that completely removed James while preserving the background and the new
generated Linus
Not new and improved just new and generated. I see you there and I
See a result
But I'm honestly really proud of I can't believe what the team accomplished here
Like it's not perfect. James could have spent a little bit more time mastering my cat-like grace
but
Given that this was our first attempt and we learned so much doing it not to mention had a ton of fun
I am glad that we completely blew our time budget for this project
Because hey
At least we had Monday comm to help us keep that under control
Monday comm made it way easier to keep track of tasks recruit actors and delegate assignments across departments Monday
Comm is built for the way you want to use it with tons of customization. You can import any data
You need track progress among team members and automate tasks so you can focus on getting work done
Don't take my word for it though. Give it a try with a 30-day free trial at Monday
Comm when you sign up today at the link below
Which is Monday comm it's really easy when your service is the same as your URL Hyundai comm, right?
It's easy to find guys if you haven't seen it already
The finished fully deep faked video is up on our short circuit channel. So go have a look. Thanks for watching. Is this really me?
You'll never know