>> This is Baruch.
>> This is Aaron.
>> This is.
>> The Open Source show.
[MUSIC]
>> I'm Aaron. I'm a Cloud Advocate at Microsoft.
>> I am Baruch, I'm Head of Developer Relations at JFrog.
>> Alright Baruch. So we're talking today about Go Modules.
So both of us had a ton of involvement in this space.
>> Well, you're the head of the Athens Project.
>> Yep, and you work on GoCenter at JFrog.
Go modules came about from
dependency management needs in the Go community.
>> Right. Because originally Go
didn't really have a dependency management. That was
an area that would left for the community to figure
out which the community did pretty successfully.
If I count correctly,
we had about 17 different dependency management tools for Go.
>> All of them did something well and then they
had some lacks too and kind of each one.
>> I would say the biggest lack is that they're
all worked about this concept of vendoring.
>> Vendoring was pretty tough because you had
this source for your dependencies
and that source would be like in GitHub,
GitLab or Bitbucket or something like that.
Then if you needed one of those dependencies,
one of your vendoring tools will just pull it
down and you'd have no idea where it came from.
You'd have no history from the VCS or anything like that.
>> So it's basically you
took someone else's code from their source control,
detach it from history metadata and shove it into
your source control performing a very bad flavor of forking.
>> You would check that code in right?
>> You checking if your own source control
you detach it completely from history,
half a year from now you have no idea where it came
from and there are issues like you can never update.
>> So we're talking about a blob of code on
your machine you probably don't trust.
You don't really know where it came from.
You just know this is code that does what I need to do from
my app and as a package maintainer
you have no idea who is using your code either.
So if you make a change and someone decides to upgrade,
like pull the new code down,
you have no idea if you're breaking
someone's code base or introducing
a security flaw or maybe even fixing one of those security flaws.
We live with that for a while,
we made some progress in the vendoring space but ultimately,
the Go core team came out with
this new dependency management system that's
built into the Go toolchain. It's called Modules.
>> It started as a couple of GoDev experiments, godep first and
then vgo and that was on the road towards 1.11.
Now we have Go Modules built into the language.
So let's talk about that.
>> Go Modules has a dependency resolution algorithm.
The thing that I think both of us are really excited
about is this download protocol and this is the first.
So if you do Go get and Go 1.11 or later.
Now Go get knows how to talk to
HTTP servers and not just VCS like GitHub,
GitLab and so on.
>> Now instead of having the sources of your dependencies,
you can actually have modules.
You can have archived pieces of software which have
metadata on what they relate to and they have
a version to look at and to understand how it came from.
But what about vendoring?
Where do we store all this?
Is it still like unzipped and stored as sources?
>> So one of the cool things I hear you
talk about all the time and it really
resonates with me is this idea of a multilayered cache.
So you were talking about
sort of a local cache but it's not vendoring.
It's like an ephemeral cache on your machine.
>> We don't commit
those dependencies to our version control anymore.
It's just place that we won't
download it really every time we build.
We just have a cache on our local machine
in go path actually when we used to have our vendoring,
now is just a cache of those dependencies.
>> So on a local cache,
can I just delete that at any time?
>> First of all, you can delete it anytime
but also if you have a team member that
needs to run the same build because they have
the same sources and the same dependency declaration,
they will have to download the same dependencies
for their machine as well and we want to speed up to that as well.
>> That's that organizational cache.
>> Exactly.
>> So from my perspective,
I started this thing called the Athens Project.
That is where that comes in.
One of those HTTP servers that we call
a module repository and you would run that
internally to your organization and it would
be able to not only serve modules
for your private code but also talk
to something that Athens calls an upstream.
So tell us about what an upstream could be.
>> Before we get into upstreams,
I probably should mention JFrog Artifactory,
it also supports Go and it is exactly what the project Athens is.
It's an organizational cache,
universal but it also supports
Go and that's very important because it
provides speed for everybody resolving but also immutability.
Once you cached your artifacts
once in Athens or in Artifactory, they are there.
They are in your repository.
You can pack them up and the entire organization can enjoy it.
But what happens if they're not
there or if you just don't have an organizational proxy?
This is where the central repository comes in.
Go central repositories are different
because they are actually tied to
the same concept of having
source codes with tags in the source control,
GitHub or whatever it is.
So we can actually build the same package from
the source and then have it in
this or other central repository and actually,
Russ Cox, the maintainer of Go has this vision of
federation of central repositories and
GoCenter is the first of them but definitely not the last.
Then it's not a single point of failure.
You can switch from one to another.
So we have all the benefits.
We speed up our resolution because
we don't clone the sources anymore.
>> Essentially, you cache them but the cache never
expires and there's no evictions or anything like that?
>> The speed, that's one benefit but the other and it's even more
important is this immutability of builds that I already mentioned.
So let's talk about that for a second.
>> Before in the vendoring world,
we had this issue where,
if you pull some code down and then you decide,
oh I want to upgrade that dependency.
Well, you might try to upgrade it a month later and guess what?
The maintainer of that package,
maybe they just deleted the entire repository or made it private.
>> What's really scary is if someone deleted
their organization username and repositories entirely
from GitHub and someone else created
the same dependencies with the same name
under the same version with completely different content.
>> It's a silent failure.
You pull down the new thing and maybe
your code base might even still build and that
one's really insidious because now it
might do something completely different.
So immutability, this is
the big advantage of both Athens and Artifactory and GoCenter.
Then any of the future ones that we can federate
around the entire ecosystem because the minute that
they download one of those dependencies and put them
in their cache or their storage, they never change.
>> Of course and now we have these hierarchy
of caches that you can trust to a certain degree each
and every one of them you know exactly what comes from
where and then you can build this system that will make fast,
reproducible, and secure builds for Go.
>> All right. So we've got
documentation for Athens that I will put in the show notes.
How do we find out more about GoCenter?
>> Gocenter.Io.
This is where you go. Very important.
You are all on the Gophers slack obviously,
there are channels for
Athens and GoCenter and we are there as well.
>> We are always there. So check out all the links
below and make sure to go check out opensource.Microsoft.com.
If you like what you see today,
give it a thumbs up and also
subscribe to the channel. Thanks, everyone.
[MUSIC]