Hi, and welcome to 'This is My Architecture'.
I'm Ziv, and I'm here Ofir for MyHeritage.
Hi Ofir.
Thank you for joining.
Hi Ziv, thanks for inviting.
So Ofir tell me about MyHeritage.
Sure.
At MyHeritage we have the unique vision of connecting you
to the most important people in the world, your family members.
You start by creating a simple family tree
by taking a DNA test, and we'll do the hard work of matching your data
with our huge database containing billions of family tree profiles
and millions of DNA samples.
With it, you can make amazing discoveries about your family history.
Nice.
And what are you going to talk about today?
Today we're going to talk about Deep Nostalgia,
a feature that became an internet sensation immediately after its release.
In Deep Nostalgia, you are asked to upload an image of your ancestors,
and you allow us to bring it back to life.
We take the static face in a still image,
and we make it move, blink, and smile.
And this feature was so beautiful that it made millions
around the world upload photos of the loved ones
and some of them to cry after seeing their beloved family members smile for the first time.
This is great, it's an amazing feature.
I remember when it went public.
Let's dive into the architecture.
Show us how you built it. Sure.
So when our user wants to animate his photo,
he reaches to our API server,
which is deployed on an EC2 Instance.
And at this point we generate a pre-signed S3 URL,
which the user uses directly to upload this image to S3.
From there the animation process starts.
We generate a request to our SQS queue
and our GPU fleet of EC2 Instances consume this message from the queue
and do the hard work of animating the image
using deep learning techniques.
So here all the magic happens.
How do you manage this fleet of EC2 Instances?
This fleet is managed with an auto-scaling group
that allows us to scale in and out according to the current volume of traffic.
Actually, we have GPU instances, which have a basic on demand capacity
and Spot Instances right on top of it.
The scale is controlled by the number of input messages in the main queue.
So you are using Spot Instances as well.
Great, and I see that you designed asynchronous architecture here
and you have this SQS in the middle, why did you choose that?
This feature is asynchronous by nature.
It takes several seconds to animate an image,
and we don't want to keep the user delayed.
Moreover, this architecture allows us to have a highly available feature
with fault tolerance. Great.
So in case of a failure or Spot interruption,
another server can pick up the message, and you never lose a request.
This makes sense.
And while the animation is completed, what happens next?
How do you let the user know about it?
Yeah, well when the animation completes
our GPU instances store the resulted artifacts in the target S3 buckets,
and they send a notification to SQS
letting us know that it completed the status of the animation.
Now our state manager consumes this message from the SQS queue
and updates our state manager with a result of the animation.
The user continuously pulls the status from the API server
and when he understands the animation completes,
he can download the artifact using signed CloudFront URL.
Now, if the user wants to go even further and share this with his family members,
most likely they will hit the CloudFront cache on subsequent requests.
So you mentioned that the feature went viral really fast,
and you had a lot of traffic.
What scale are you talking about here?
Yeah, in the first two weeks after the release,
over 50 million animations were created by our users.
This is a great scale.
And you're using Spot Instances for animation servers.
How did you handle this demand of Spot capacity?
Right.
At this Spot, we were actually using a single region,
let's call it our main region, which handled all of our traffic.
We're already using multiple types of GPU instances,
and we're already spreading it on multiple AZs.
Now, as scale rises, we wanted to even leverage further
and take instances from other AWS Regions.
So what we did is we took our animation services
and deployed them at five more AWS Regions.
And then we were able to scale further and use capacity pools on other AWS Regions.
Now from this point, we need something to actually route the traffic to other regions.
So we added the Lambda here, which grabs the message from the input queue
and that's the work of routing the traffic to AWS Regions
according to the current Spot capacity.
So at this phase we have a local SQS here
allowing our GPU instances in the main region to pull the message from here,
and now we don't even need to pull it from the main region.
I get it.
So you could fetch messages from other regions directly from the main queue.
Why did you choose to add this Lambda function in the middle?
Right, the thing is that Spot Instances are dynamic by nature,
and we wanted to control the portion of traffic
that we send every region in this Lambda over here.
Moreover, we wanted to process as much animations as we possibly can
in this main region because the source and target buckets
reside in the main region, and we wanted to avoid inter-region traffic costs.
I see. So you could also control the cost here.
Did you consider any other cost optimizations here?
Yes.
We focused on our animation services,
which is the major cost aspect of this architecture.
We actually wanted to see if we can use the better GPU instances for them
as we started back then with P3 Instances.
And recently AWS released G5, which is a newer GPU generation,
and we tried it out.
It turns out that while it gives us pretty much the same performance,
we were able to improve cost dramatically
leading to a significant cost saving.
Thank you Ofir for sharing this architecture with us.
Thank you, Ziv, it was a pleasure.
And thank you for watching 'This is My Architecture'.