Welcome to 'This Is My Architecture.' I'm Benjamin from AWS.
We are here in Tel Aviv, Israel.
Today I'm joined by Tom from Wix. Hi, Tom.
-Hi Benjamin. -Thanks for joining us.
-Thank you for having me. -Tom, tell us a little bit about what Wix does.
As you know, Wix is an online presence platform serving about 5% of the world's websites
with over 700 million unique users per month.
Okay, so right off the bat you're operating at quite a bit of scale.
-Yeah, it's high scale. -And today we wanted to talk about your SLED system.
-Tell us what SLED does. -SLED is an end to end testing framework
we developed here at Wix and it's mostly based on AWS Lambda infrastructure.
Okay, so testing to test for production changes to your websites.
What kind of scale are we talking about for those?
So let's say the average rate for production changes and deployments,
I'm talking about unique deployments, is about 400 a day.
All right, so 400 deployments a day you're going to be testing automatically.
You're going to be testing all the time. All right, so how do you test on SLED?
So, the first thing we do is SLED is already installed on a laptop or the CI machine at build time,
it doesn't really matter. The first thing we do is that
we execute the test.
Let's say that the developer writes a simple test, an end to end test using Puppateer in Jest
and good and popular open source libraries.
The first thing we do is we want to make sure that the user that just invoked the test
is actually a Wix developer and it's authenticated properly.
So what we do is that we use AWS Cognito along with the built-in support with Google Sign in .
So we make sure that every developer is indeed a Wix developer.
So that's the first thing and we know the user is authenticated.
Okay, so authentication. What happens next?
So once we're authenticated we look at the test code, the test code you've just written.
We take the test code itself, with all its interdependency,
and we pack it and compress it and we upload it to S3
with a specific unique ID we give, let's say it's XYZ.
And that's what we call the execution bundle.
-Okay. -So once we've uploaded the execution bundle
what we do is that we start running the test with the native testing framework.
In this case it's Jest.
And what we do is we use Jest API and tweaked it and now instead of running the test on the laptop itself
what we do is that we start invoking Lambdas, not running the test like it should have been run.
What we do is we start invoking a lot of Lambdas.
By the way, the average test we have in the major builds in Wix
is about 700 different test files.
When I say 700, it's 700 different test files with different Chrome and different Puppateering instances,
and we start invoking 700 of this in parallel, all at the same time.
So, using Puppateer and Chromium you're able to fake a browser
running on Lambda, correct? -Exactly.
And I believe you're using Lambda layers as well?
Yes, we use layers, we actually use only two types of layers.
It's very easy to maintain and keep track of the code to make sure that the run times look the same
both on the laptop and on the Lambda.
We have one layer with pre-installed Chromium AWS Lambda.
Again, it's an open source, very popular library
that allows you to install Chromium with all it's optimizations and tweaks,
and have it run smoothly along with Puppateer on a Lambda run-time environment.
The second thing we have is just the standard Jest in its interdependency.
We pre-build and we pre-install it on the Lambda itself.
So every time we invoke a Lambda usually the Lambda is already warmed up
because you see what's our scale.
We already have a Chromium instance ready to be executed along with all the Jest dependencies.
So the only thing we need to download from the Lambda, and that's the second part.
What we do is that we pass it, the execution bundle ID,
the Lambda downloads immediately the execution with the test file
just brought from the developer laptop and we start executing smoothly
as if it was executed here, only it's happening 700 times all at the same time.
So every one of these executions has its own sort of headless browser.
It is executing a test against your production website, correct?
Exactly. Pure production with the minor changes that the developer just did for its own personal code.
And then once its figured out sort of if the execution failed or succeeded,
how does it report back?
So, because it's just a standard normal Jest installed on the Lambda
what we do is we respond to the regional invocation
with a simple JSON.
And that JSON has everything needed in order for the test originally running on the laptop
to determine whether it passed or failed and if it failed, what were the errors and expectations
not being fulfilled.
So what we have is we have a developer right here.
Again, it could be CI the same thing but we get a very, very smooth
and very cool developer experience where you just invoke 700 different browsers
on his laptop in a matter of let's say, 2 or 3 minutes.
That's it, for 700 different tests.
So it's almost like the laptop or the CI machine doesn't even know
that you were running these tests in parallel or you were running them remotely.
-It's just happy. -Exactly.
How do these two fit in?
So, because we consider ourself as a full blown infra,
what we can do is that we can monitor and report back to DynamoDB and CloudWatch
and better understand the infrastructure and maybe some tweaks and optimizations
needed in order to make sure that our up time is as high as possible,
and every test is being executed properly and returned properly with the correct message.
So give me a sense of how many of these tests are you running in general.
So I just looked last week. In 5 working days we have 1.2 million
different SLED tests, different SLED test executions. Meaning 1.2 million browsers executed
on Labmda, reporting back and playing around with our production system.
So if you wanted to run this on Lambda and you wanted to run it say in like your CI server
or one of the developer's laptops, you probably wouldn't be able to do it?
You can try executing 700 browsers on your laptop and see how it goes.
And 2 million executions a week. That's amazing.
So, I really like this and I like it because you're using Lambda in a smart way.
You're sort of faking the testing environment that you're running on your CI servers, your developers.
You're giving them, sort of very seamless, smooth developer experience.
They almost don't know that it's happening.
But you're benefiting from this endless compute.
This is a really cool use of Lambda and clearly you're running at a scale
where you need this to be robust, you need this to be stable,
you need this to be monitored so you're using the platform to monitor this system well.
So this is pretty cool and thank you for sharing it with us.
Thank you for having me again.
And thank you for watching 'This Is My Architecture.'