[MUSIC PLAYING]
BRANDON DEROSIER: Hi.
I'm Brandon.
I'm a software engineer working on Flutter Engine.
LEIGHA JARETT: And I'm Leigha, a product manager for Flutter.
If you tuned into Flutter Forward
or you've been reading our release blogs,
you've probably seen the name Impeller thrown around
quite a bit.
But what actually is Impeller?
And how does it work under the hood?
Well, today you're going to find out.
Impeller is a new renderer within Flutter's Engine.
Until now, Flutter has been using something called Skia.
The problem is that Skia wasn't designed for Flutter.
It has a ton of rendering features built
for a wide range of devices, which
means that it's not always optimized for Flutter's needs.
Enter Impeller, Flutter's new renderer.
We built Impeller to specifically focus
on the rendering needs of Flutter applications.
And our main goal was to eliminate
jank or any stuttering that's happening inside of your app.
This way it'll always look and feel great for your end users.
Now for everyone who's watching who's not a graphics engineer,
you might be wondering, what's a renderer.
Well, a renderer is software that
helps you translate your UI code into the pixels
that you actually see on the screen.
So say you have a simple Flutter app like the one
that I'm showing here.
In the Flutter framework, your tree of widgets
is backed by a tree of render objects.
Render objects contain the instructions
for how to actually layout and paint the widgets.
These instructions are given to the engine
and stored in an ordered list of simple commands called
a display list.
And now this is where things get interesting.
The engine is responsible for using
one of the available renderers, either Impeller or Skia,
to draw this display list to a surface texture,
or a grid of pixel values that can be displayed to the screen.
It leverages the GPU by setting up
a collection of things called render pipelines that
can be used to render everything in the display list.
So let's dig into the details.
Before we can use a render pipeline,
we first need to take all the paths drawn by the display list
and tessellate them into sets of triangles.
Then each vertex or point on the triangle
is passed through something called a vertex shader.
Now a shader is just a small piece of code
that gets executed on the graphics device.
And in this case, the shaders are internal,
meaning that they're a part of the Flutter engine's code base.
But us Flutter developers can author our own shaders
too so more on that later.
So the vertex shader takes the vertices
that make up the Flutter logo and it moves them to the place
on the screen where we actually want to draw them.
And now the Flutter logo is in the right place
and we begin iterating through all the triangles.
So let's zoom in a little more.
For each triangle, we figure out the specific pixels
that are inside it.
And this is called rasterization.
Then a few checks are run for each of these squares
to determine if we actually need to compute a color for it.
For Impeller, these checks are really
important so we'll be coming back to this in a bit.
So now all those pixels that are in the triangle
are pasSED through a fragment shader.
A fragment shader is another snippet of code
but this time it takes the vertex shader's output
and computes a color.
And like I mentioned before, Flutter developers
can override this by creating our own shaders
to create really neat effects using the fragment program API.
Anyway, back to our render pipeline.
Lastly, the output colors are blended with the color that's
already been drawn.
In this case, our blending was really simple
and we just replaced the color.
And that's how things get rendered on the GPU.
Now over to Brandon to talk through how
these render pipelines are actually
implemented in Impeller.
BRANDON DEROSIER: Thanks, Leigha.
Impeller has a layered architecture
where each layer uses the one below it to accomplish its job.
Before, Leigha showed an example of how the Flutter logo
widget gets converted into a display list.
In this example, the display list
just contains a few operations to position it
in the center of the screen along with four draw path
ops, one for each of the quads in the logo.
So let's take this display list and draw it using Impeller.
First, all of the display list operations
are dispatched to something called Aiks.
Aiks is the topmost layer of Impeller.
And it mainly consists of a canvas drawing API.
By the way, the name Aiks is just Skia in reverse.
The job of Aiks is to take high level commands
like Draw Path and Draw Image from display list
and turn them into simpler, self-contained drawing
operations called entities.
And this is where the next layer comes in,
the entities framework.
In our example, the four draw path ops from the Flutter logo
each end up getting an entity of their own.
Each entity contains a bunch of properties
that every single drawing operation needs,
like a transformation matrix that
encodes the position, rotation, and scale.
Every entity also gets a content object
assigned to it, which contains the actual GPU instructions
needed to draw the entity.
There's a contents for drawing solid colors, images,
gradients, text clips, everything
that Flutter apps can draw.
Sometimes we even have multiple specialized contents
that draw the same thing using different algorithms
with different performance characteristics.
That way Aiks can pick and choose
the most efficient rendering algorithm
depending on the situation.
In this case, all four of the Flutter logo's quads
are just filled with solid colors
and so each of their entities get a solid color contents
assigned to them.
Now we need to spell out these instructions in a way
that the GPU can understand.
Because Flutter apps can run on a bunch of different platforms,
Impeller needs some kind of translation layer
to communicate with the GPU.
This is where the layer below the entities framework
comes in.
I'll refer to this as the hardware abstraction layer.
This layer is just a thin abstraction
that talks to the graphics driver
through various standard graphics
APIs like Metal on iOS and Vulkan on Android.
Now we're ready to start telling the GPU exactly what to do.
Remember those render pipelines Leigha mentioned before?
This is where those come into play.
Each contents uses the hardware abstraction layer
to draw itself by instructing the GPU to execute the render
pipeline containing the shaders we actually want to use.
And really that's it.
Those commands are executed using the graphics API
and then the resulting texture gets displayed to the screen.
But there's one huge detail I haven't covered yet.
Those shaders and the render pipeline
also need to get compiled down to instructions
that the GPU can execute.
And this process is very expensive.
In Skia, this compilation process
happens at runtime right on the frame.
The pipeline actually needs to be used to render something.
This usually causes the frame to go way over budget,
resulting in a noticeable stutter.
We often refer to this problem as shader compilation jank.
Impeller vastly improves on this issue
by performing the most expensive part of this compilation ahead
of time.
And this is where the last component of impeller comes in.
When the Flutter Engine is built,
all of Impeller's shaders get compiled
into bundles using Impeller's offline shader compiler
called Impeller Scene.
In walking through Impeller's architecture,
you've seen that Flutter really has complete control over how
it renders graphics.
Flutter isn't subject to the architectural or design
limitations imposed by platform-specific UI toolkits
because Flutter doesn't even use them.
Instead Flutter's engine talks directly
to the graphics driver.
This is what allows you to build rich, unique apps that
behave exactly how you want on any platform.
This lack of dependencies also makes Flutter a great solution
for embedded use cases like automotive dashboards, IoT,
and digital signage.
LEIGHA JARETT: Now that you have an idea of what's
happening under the hood, let's highlight
some of the key architecture decisions
that we made that makes Impeller a great renderer for Flutter
applications.
So like Brandon mentioned, one huge benefit
that Impeller offers is eliminating jank
from compiling those programs called shaders.
To do so, Impeller doesn't generate shaders
in the same way that Skia does.
Instead impeller has a set of handwritten shaders compiled
in advance.
But you might be worried that initializing
a bunch of precompiled shaders could mean slower startup times
or bigger Flutter app sizes.
To prevent this, Impeller uses alternative rendering
techniques that leverage a much smaller, simpler
set of shaders compared to the many specialized shaders
that Skia dynamically generates.
Another reason why Impeller offers great performance
is the way we implement anti-aliasing.
Anti-aliasing is the art of smoothing out
jagged edges of drawn elements so that they look more natural.
Impeller uses something called MSAA
to perform this work, which is present across all devices
supported by Flutter.
It's pretty cheap to perform.
And it produces excellent quality results
on modern mobile hardware.
Another thing we focused on is how
Impeller implements clipping.
When rendering widgets, Flutter frequently
uses clip masks to cut out shapes
so it's really important that this process is efficient.
Impeller specifically takes an approach
that leverages hardware features to ensure
that clipping is super fast.
Remember earlier when I mentioned those special checks?
One is for something called a stencil buffer.
Now we won't go into too much detail
but basically, as Impeller draws the pixels for the clip thing,
it tells the GPU to check the stencil
and filter away all the pixels that aren't
part of the filled shape.
This means that even if a clip is really complicated,
like the ones used to create these kaleidoscope animations,
it's still a pretty cheap operation
so we can see some really huge performance improvements.
On the left hand side, we see Flutter pre-Impeller.
And it's rendering at about seven frames per second.
And on the right side, Impeller renders
at about 60 frames per second.
BRANDON DEROSIER: So we just talked
about some of the design choices in Impeller.
For most Flutter developers, this
means you'll see more predictable frame
times when you use Impeller.
But for those that are interested in more
advanced rendering use cases, Impeller also
aims to offer a launchpad that allows you to solve
for other graphical needs.
For example, earlier this year at Flutter Forward,
we showed off a proof of concept 3D scene graph built directly
on top of Impeller's hardware abstraction
layer called Impeller Scene.
Impeller Scene has its own contents in the entity's
framework and it draws 3D objects
using the same kinds of render pipelines
we talked about earlier.
Now there are tons of consequential decisions
that go into building renderers.
And while we can certainly build 3D renderers that are useful
and make smart tradeoffs, no one renderer
can solve every use case perfectly.
And so as a next step for this, we're
actively experimenting with ways to surface more control
over rendering in the Flutter framework
so that anyone in the Flutter community
can build their own rendering packages
and create custom render pipelines that
integrates seamlessly with the rest of the widget ecosystem.
Impeller's scene is a proof of concept
and a showcase for the kinds of developer
experiences we're seeking to enable with this effort.
With that in mind, let's take a look at the demo widgets.
Basically, everything is centered
around a special widget called scene box.
Scene box takes a node which can be created
from a 3D model that's added to your Flutter app's asset
bundle.
Nodes can be composed together to form a tree just
like normal Flutter widgets.
Here I'm wrapping the asset with another node
that applies a rotation.
You can add the same asset to the scene as many times
as you want.
And Flutter will know to only load the asset once and reuse
it to inexpensively render copies,
kind of like how image assets work.
And you can even play animations built
into the asset that were authored using 3D modeling
software like Blender or Maya.
And since Scene box is just a regular Flutter widget,
any of these node or node animation properties
can also be animated in exactly the same way
that you would animate anything else in your build methods.
And of course, hot reload just works.
Simply export the file from your 3D modeling software
and Flutter does the rest.
With just these tools, it's possible to construct
rich interactive scenes composed from many different assets.
Again, this is the kind of fluttery experience
we're seeking to enable as we surface
more control over rendering thanks to Impeller.
LEIGHA JARETT: So we just gave you an overview of Impeller.
And we went under the hood to show just
how Flutter renders your widgets to the device screen.
But that's not all.
Today we're super excited to announce that Impeller is now
the default renderer on iOS for the stable version of Flutter.
So upgrade Flutter and try it out.
And if you want to see Impeller in action,
you can install G. Skinner's Wondrous app.
Right now Wondrous uses Impeller on iOS.
But we're looking forward to the next version of Wondrous
which will include Android support for Impeller.
We're really excited about where Impeller is headed
and the new rendering capabilities
that we can start bringing to Flutter and we hope you are,
too.