>> I'm Lachie.
>> And I'm Oliver.
>> This is The Open Source Show.
[MUSIC]
>> I'm Lachie, a PM at Azure.
>> I'm Oliver Gould, the creator of Linkerd.
>> Oliver, in our previous episode,
we learned all about Rust.
Let's actually learn how Rust is being used in the real world.
Can you tell me how you made
the decision to write Linkerd 2 in Rust?
>> Yes. So when we actually started working on Linkerd
in 2015 or so,
we tried to do the first implementation of
Rust before it was 1.0, it was still in beta.
We got about a week or two into it and it was just too hard.
The ecosystem wasn't there.
There wasn't a networking stack,
there weren't HTTP libraries.
We were going to build everything from the ground up.
So we didn't. We stopped and re-wrote it all in
Scala and a couple of years passed,
and all the sudden Rust had
all the things we wanted to have or almost did.
So we started to invest very heavily there and built
Linkerd 2's proxy in Rust and
that's been off to the races ever since.
>> What was it about what you were doing in Linkerd 1
that led you to look at languages like Rust?
>> In Linkerd 1, we had a lot of production users at scale,
and it was good if you could give the
JVM enough resources to be healthy.
But when you're running
containerized applications with hundreds of instances per host,
running a side curve per each container in
a JVM world just isn't feasible memory-wise,
and then performance-wise, latency-wise,
The JVM is a garbage collector which introduces
slowness randomly in your system.
With Rust, we have the compiler deal with all of this.
So we don't have a garbage collector at runtime.
The compiler deals with safety
making sure that we neither have memory leaks nor do
we have to go do a bunch of memory management at runtime.
So when a request is done using memory,
resources just get freed,
and we have a much healthier system but Rust is designed with
concurrency and performance and safety in mind from the ground up.
We don't want to depend on Open SSL.
So being able to move into an
entirely Rust-based system means that we get
a much higher degree of trust in the way
memory is used in very fundamental ways in that part of the stack.
>> So describe to me the function of
this proxy that you reference?
What is it responsible for?
>> So Linkerd is a service mesh.
What our Linkerd proxy does is it gets deployed with
applications and manages many of the reliability, performance,
and security concerns too.
So applications don't even take those concerns internally.
>> As you compared what you've done in
Linkerd 1 to Linkerd 2 and Rust,
performance-wise, did you notice some changes?
>> Once we've worked out some of the kinks in
terms of tuning socket options.
It far exceeded my expectations.
In Linkerd 1, we were probably
several milliseconds p99 and 100 mgs or more RAM.
Two we're about two mgs of RAM, and submillisecond.
So it's actually substantially better.
>> Substantially better.
>> We looked going to Linkerd 2 that we
want to really kill the resource issue.
We want to keep all the correctness that we got
with Linkerd 1 and the type system in Scala,
move that into a native environment where you can have
no memory overhead of
garbage or garbage collection or anything like that.
>> What was it about Rust specifically
that made you really comfortable with
putting it in the datapath at an early stage of Linkerd 2?
>> We wanted that native performance.
So the performance couldn't be a sacrifice.
We wanted to get correctness which the type system gives us.
But we also wanted the trust of not linking against C libraries.
>> So having a component written in Rust
that's responsible for such a critical piece of infrastructure.
It begs the question,
is Rust production ready?
>> I think production ready is different for different people.
>> Yes.
>> We've been willing to make
the investment to make it production ready for our needs,
it may be harder to scale an engineering org with
hundreds of people up on that in a production ready way,
but for our small team working on Linkerd it's been really good.
We were actively investing on lots of diagnostic tooling
and tracing systems that we're going to have to
build into the ecosystem.
But it's actively becoming a production ready stack.
>> Okay. So is there an ecosystem
of crates that you can pull out there from
the wild that is solving boilerplate problems?
>> So if you go to creates.io you can look
through a whole catalog of various crates
sorted by popularity and so on that deal with crypto,
without C linking for instance or that deal
with caching or that deal with HTTP caching.
All sorts of different crates
out there that make it easy to get up and running.
However, a lot of the networking stack we've invested in.
So things like Tower which is
the Finagle like service abstraction. On top of that,
we've built gRPC systems, that's HTTP2.
We actually had to write in HTP2 codec and Rust.
So in our networking stack,
we're not using any C libraries.
We're not linking into
levy or something else than just running with it.
But luckily, it's been used in a lot of places.
So it's easier to get that testing now.
>> Let's talk about the developer experience around Rust.
What is it like for your development team?
>> So Rust comes with its own set of build tooling,
it comes with cargo by default which is
the easiest way to get up and
running with packages which they call Crates in Rust.
So I for instance switch between Go and
Rust in VS code all the time other people on the team,
use other systems, but because
it comes with a Rust language server that does
it independent from editor so you can get
really good editor integrations across editors.
>> How do the development team find
Rust package management as compared
to languages that they were using, Scala or such?
>> So unlike Scala,
Rust's packaging system is source-based.
So I'm always downloading sources and compiling
but I'm not vendoring them into my code in the same way.
In Linkerd, we git.lock file,
a cargo grit.Loc which pins us on dependency.
So all developers have
the same set of sources when they are developing,
and we can move safely.
>> What about testing and test frameworks.
How different is that in Rust?
>> It's basically like any other modern language that has testing
built-in from the ground up so
Cargo has cargo test and it just works.
You can do things like fuzz testing with quick check which
we do a whole bunch of and we write
integration tests that do network tests with Rust.
So it's totally viable.
>> How are you tracking something like Linkerd against
all the movement and Rust is that
a challenge at this point in the development?
>> Luckily the Rust community divides
their releases into three channels.
We have stable, beta, and nightly.
In Linkerd, we pin to stable.
We're very conservative in terms of the new features we adopt,
when you're starting and Rust must people pinned to nightly.
Nightly, sometimes breaks but
that's where all the cool features are.
We make investments and playing with the
nightly so that we're not surprised by anything when it comes in.
We want the community because there's
no vet them before we pull them into the project
>> Okay. So if I have a team we want to write something in Rust.
Where do I go to get started?
>> You can go to rust-lang.org but
my suggestion is to go find a project that's
written in Rust where you have a community eco-system
to go read and get
involved and probably easy to start that way.
>> Okay. Thanks for joining us and
teaching us more about how you've
used Rust to solve problems in Linkerd 2.
>> Thanks.
>> All the links are in the description,
and you can go to opensource.Microsoft.com.
If you like this video, give it a thumbs up and subscribe.
Thanks for joining us on The Open Source Show