>> Hi, we're back with Mucku to talk about
Frozen Objects in this episode of ON.NET.
[MUSIC]
Hi friends. Welcome to the next episode of the ON.NET Show.
I'm Moni, the owner of the.NET Core garbage collector.
>> Hey, I'm Mucku, I work on the Bing performance scene.
>> So Mucku, last time we
were going to talk about Frozen Objects, right?
That's one of the features that you are
using during your startup stage.
So frozen objects had existed in CLR for a long time.
I mean I'm talking like it started to appear in
CLR 2.0 not.NET Core, but.NET 2.0.
>>.NET 2.0.
>> Yeah. So why did we have this feature?
Well, if you observe that some objects are immutable.
They don't change after you create them,
and very likely that they
would live throughout the process lifetime.
So it would be nice if we could save the work
to have to reprocess them every time we say do a Gen 2 GC, right?
Because a large part of
your Gentoo consists of these immutable objects,
we want to optimize that.
So we introduced this feature called Frozen Objects.
In the GC's perspective they actually live
on segments that we call read-only segment.
>> Right.
>> So normal GC heave lives on
read-write segments because you modify them.
But read-only segments are
for users to put their frozen objects on.
We use this on frozen strings.
I think that's the only object type we actually use this on.
So can you tell me what in
your scenario that made you want to use this feature?
>> Yeah. When I found out
about frozen strings like in [inaudible] two days,
obviously much later, and then I tried them and I was like,
oh, it'd be great to try them in
the next versions of Dyna Framework.
I think has been if deaf out or has
been disabled in the actual.NET Frameworks after 2.0.
But in.NET Core, we sort of were like, hey,
we'd like to use it in the use case for us
is that we have these like any files,
there are these large called them in-memory database read-only.
>> Right.
>> It's essentially like configuration data,
other sorts of things that we never change or we'll
change either after a process is torn down,
essentially read-only, and I was
thinking it be great if we could actually have
all of these objects which have to
be read from this any file as strings,
parse them, do all this work at startup.
Essentially to make them never be able to change them.
So I was thinking wouldn't it be cool if we could use this,
and this can get large like multi-gigabyte.
>> Right. Yeah. It would be nice if GC
doesn't need to reprocess them.
>> Reprocess them, yeah.
>> But the cost of read-only objects or read-only
segments in GC is very little compared to read and write ones,
because all we do is if we observe that there are in range,
in range means if they're in the GC heap range,
and we have a lowest and highest address
for the GC heap and then if
a segment of read-only objects are
frozen objects happens to land in this range,
we have to basically materialize the GC bookkeeping for
it because you might be using the mark beds on its own right nods.
When GC actually happens,
all we do is we would naturally
mark there and then you just clear the mark beds.
That's all we do, is it's very cheap compare.
So what you just described sounds like
the perfect scenario for using the frozen objects.
Of course, if you want to use
this feature you have to somehow construct these objects.
They can't appear in thin air.
So you have to construct to add and historically,
we just constructed them in the runtime itself, right?
Runtime would go make frozen strings somehow construct them
and [inaudible] and I put them in
the image and then when the CLRs load that we'll load them up.
>> Right.
>> Basically, all they do is they register
that memory region as a read-only segment to the GC, right?
>> Correct.
>> But you did those completely without
the runtime changes and it's all in managed code.
>> Right.
>> So that's very cool. So let's talk about that.
>> Yeah. So in
CLR2 or these when you have this thing
you would basically be only be able to do strings.
I had to use engine which is somewhat limiting.
So for our use case neither of these came.
We're running on.Net Core 3 and we want to be able
to actually use these for all these different kinds of objects.
They have lists in them.
Strings in them, all sorts of complicated objects.
So actually creating them is sort of interesting.
The code is actually a managed NuGet package,
and you can add this to your application.
>> That's right. Anybody can go and download.
>> Anybody can go and add downloaded added
to their application and essentially you
think about this as like a deployment or a built post build set,
you've got your application and you want to use.
You want to make these frozen objects.
>> Right.
>> You run your application, you point the frozen.
We call this a frozen objects utilizer.
You say, hey, I have this object and I want to just
serialize this to disk as a frozen segment.
>> Another cycle, another step in your deployment.
>> Another step in your deployment process.
You basically do this and essentially voila,
you have these objects from non-contiguous memory on the GC heap.
>> Yeah.
>> You get contiguous
essentially objects on disk which is really cool
because you get locality benefits and all that stuff
from these frozen objects.
So I can now show
the sort of how we actually serialize this on disk.
>> Yeah. Let's do that.
>> So here basically,
we're going to start with like this is
an object and you can see that
the object has three references and some long fields.
We want to start serializing this object.
A thing to note is if you have this object
and has other references we have to take care of
storing the information what we call
that are not live memory references.
They're actually going to be cookies that we store on disk.
So for example that method table pointer has to
be stored in a persistent in some interesting way.
So what we start is we have this and I'm going to
show on the upper right corner is
basically like a queue and where's the state
and action being done whereas we enqueue and dequeue things,
and this is sort of like the first step in the serializer.
We go, we find these references and we say,
hey, we found object,
we dequeue Object one which was the root object that
you basically started with and we added
object four which if you look is from you have to go look it
from the left side come that object
four, object two and object three.
>> Sorry to interrupt.
So that means if I have multiple objects I want to serialize,
I can just make like an array that points to all of them.
>> Yeah.
>> So you start from the root objects.
>> Right. We always start from the root object
but if you wanted to, for example,
have thousands of objects,
you could just make an object array.
>> Exactly, yeah.
>> With all those and then give
the object array as the root object.
>> Exactly, yeah.
>> Exactly. So here,
we basically have a few things going on.
At the bottom, you have basically
the data that's been written to disk.
So Object one has been written to disk but you'll
see that thing called Reserved and there's a dictionary there.
So what we needed to make sure is,
when you have the same reference that you've encountered,
we can't actually put this,
those references have to be have referential identity.
So we can actually enqueue and write
multiple objects of the same instance onto disk.
So that dictionary helps with that.
Also with the ability to point back at
references that you have encountered as you do the Object Graph.
So here, basically we go and say this object which
is its size is 64. We put that.
Then we've discovered Object four,
Object two, and Object three in
that order and we've enqueued that in our queue.
Note that knowing its size,
it's overall size that we've encountered is actually very helpful,
because then we can do all this in one pass.
>> Right.
>> To talk about the one pass thing.
This is not actually how if
the GC would do it, it would do it differently.
It would actually go do what you would do normally
mark all the objects.
>> Yeah.
>> Then go back again on them.
>> Yeah.
>> We decided not to do that because we
want to do this a 100 percent outside the runtime in managed code.
>> Exactly, yeah.
>> We're not mutating your objects.
>> Of course. Right. I mean we
want people's feedback on this feature and-
>> It's experimental.
>> Exactly. But if this is
very helpful to a lot of people and scenarios then we could,
of course, consider putting it as a long-term feature.
>> As a feature in writing. Right. So we
basically go find the next object,
serialize that to disk and again you'll notice
that the reserved region hasn't changed,
because we have not actually encountered another object with
object references which is the interesting part here.
Then we get to the next object.
Here, we have still not
encountered any new object with object references.
So you see we haven't reserved
more space for the next set of references.
>> Right.
>> Then we find an object again
with more references and we go in and figure out obviously,
write this object but then find the reserved stuff
that is going to be there and
coming in the future as we serialize that.
>> Right.
>> Then finally, we've arrived at the final object and now
there's essentially nothing else to serialize.
So if you could imagine these objects could have been
anywhere in memory but we put them together, a contiguous on disk.
>> Exactly.
>> Which is really helpful for [inaudible].
>> Right. Yeah and all this is on GitHub, right?
>> All this is on GitHub.
>> Github.com Microsoft-
>> Slash Microsoft slash frozen objects.
>> Right.
>> It's a public repo.
>> Yeah.
>> It's all managed code.
>> Yeah. Feel free to go check out the code if you
want to the concrete code corresponds to the slide.
>> Yeah. Let us know if there's things,
like there are limitations, for example,
we don't want to serialize things into PTR,
Azure or things that may have file handles that are there.
So there's a few things that are there.
But it'll be good to get us feedback to see what to improve.
>> Definitely require feedback from customers on this feature.
Yeah. So this is great.
So when we're talking about this speech and we really
want to see how can we make this faster?
So you talked about serialization, right?
So when you deserialize,
when the process starts out we need to load frozen objects or just
whenever that registration time
you need to deserialize things into memory.
That's where we really care about the [inaudible].
>> Exactly. So we did a bunch of work as you can tell on tools.
We have decree and lookups. We're doing all this work.
>> Right.
>> But serialization is much faster.
>> Right. So all we do is we have two kinds of fix ops, right?
There's the offsets that you put in for references.
>> Right.
>> Then there's the method table pointer fix ops.
So of course, we're thinking like
how can we make those to go faster?
So if you can imagine if we
had a preferred address, we can load it there.
>> For frozen objects.
>> Right, yeah. Then we don't have to write offsets,
we just write the actual address.
>> That's one fixed op gone.
>> Right and if we cannot manage
to load or add the preferred address,
we just go to the fixed ops as normal anyway, right?
Then how do we avoid the fixed ops for methods able pointers?
It turned out that in the runtime,
there's no way to tell the type system,
where to put the type info.
You can give it a memory chunk and say,
hey, just put it here.
Which means if we can manage to
again load off for the preferred address,
we're already knew beforehand where the method.
>> Exactly.
>> So we were so excited when we found that out, right?
But we're only happy for like a day.
>> Before the security people came and said
that because these are live pointers,
the fact that you can call methods on objects,
it can get, their security can become an issue.
>> Right. There is another concern which is the unpredictability,
which means if you cannot manage to load up
with a preferred address then becomes slower.
>> Correct.
>> But personally I think that
in the last part we talked about large objects or large pages.
>> And unpredictability there.
>> Right. But I think this is a different kind of
unpredictability because with large pages,
you're going to have unpredictable results throughout the process.
Whereas with this, it's like, at this time you-
>> This is quick phase.
>> Right. Exactly, it's very limited in scope, right?
>> Yeah.
>> So we could tell you,
[inaudible] I don't know, like,
entry event log and what not just to tell you that, hey,
we didn't manage load
a preferred address so you know why that part was slower.
>> Yeah. It's different from
the large object case or large pages case. I'm sorry.
Where the idea of running
your app at 10 percent slower performance
maybe significantly worse is because you're running it for
a long time versus
a discreet phase like startups. So no, I completely agree.
>> Exactly, yeah. So this is actually one of
the other reasons why we really wanted your feedback on, right?
You can let us know what you think of this unpredictabilities.
Is that tolerable to you or not? Tell us why.
We're going to put our on Twitter links below.
So feel free to connect with us
on Twitter and tell us what you think.
We really appreciate your help.
>> Yeah.
>> We've been talking about onsite per features.
So clearly you care about [inaudible] tremendously, right?
With that, I think
diagnostic ability is another thing
that you have to care about a lot.
Because when things run slower you want to know why, right?
So we're going to wrap up for this episode.
Next episode, let's talk about
the diagnostics that we worked on together for Core 3.0.
>> Cool. Sounds good.
>> All right. Bye.
>> Bye.
[MUSIC]