all right okay then okay
so hello thank you all I really
appreciate the opportunity to be a part
of CSS comp Budapest just massive thanks
to the organizers and most importantly
you all for welcoming me into your
ridiculously beautiful and picturesque
City
my name is Jeremy Wagner and I'm a web
performance consultant for site improve
my job is to do all I can to make the
web faster for our clients and I also
write and talk about performance a bit
I'm just repeating everything or that
our wonderful emcee has already said but
this talk is about paint worklets which
is a little weird for me because I
always talk about performance which has
comparably little to do performance well
mostly we'll get there
and right about now you might be
wondering Oh Jeremy what the hell are
paint worklets and it's a good question
and since I don't want to assume
everyone knows about this here's a quick
tour of what they are and what they can
do paint worklets are part of a
developing set of specifications called
Houdini and with the api's Houdini
provides you can write what are what are
called worklets and they let you develop
expressive and stateful animations
custom layout modes even that's a
developing thing and as is the sole
focus of this talk
generative artwork using the CSS paint
API seriously so I haven't answered your
question yet and out there you're
thinking ok fine journey buying that
whatever but what even is a worklet will
you get to the point worklets are
similar to web workers but unlike web
workers worklets
offer developers lower level access to
rendering pipelines for completing very
narrowly defined but super specific
tasks and in the case of paint worklets
we're talking about artwork that we can
generate with JavaScript and control
with CSS now I know Jay s comp is
tomorrow and not today so you're
probably thinking well with what the
hell is this I didn't ask for no
JavaScript at a CSS calm woo and I feel
you as much as I love JavaScript
the web can sometimes feel like this but
I feel that this technology is relevant
here even at CSS cough because even if
you're not a JavaScript devotee or
superfan you can still use paint
worklets which have already been written
in your designs without ever having to
do more than write a little CSS here
there so they're still relevant to you
and this is possible because while paint
worklets are written in JavaScript
they're embeddable in CSS and this is
what I mean by an embeddable when you
look at this lonely background image
property you might be thinking what
that's not right
isn't background image supposed to take
a URL well that's certainly the
prevailing use case but in browsers that
support paint worklets the background
image property can also take this little
nifty paint syntax too so if you think
of the URL syntax and CSS is a function
that accepts a URL string to an image
you can similarly think of paint as a
function which accepts a string that
represents and invokes a paintwork clip
and in that worklet is where you write
the JavaScript to generate background
images that CSS can use so in this
example we're calling paint to invoke a
worklet that is registered under the
name amazing but until we actually
register a worklet to that name this bit
of CSS won't do anything and therefore
will not be very amazing and this is
what that registration process looks
like which is done in JavaScript on your
page or your main app code that runs on
the main thread because the paint API
isn't available in all browsers you'll
need to do a quick check to see if
window dot CSS dot paintwork late' is a
thing that you can use and if it is you
can then use its add module method to
load your worklet from a separate
JavaScript file and what is a paint
worker worklet really I mean what is it
it turns out it's a class with a
predetermined shape which consists of a
method called well paint and with that
within that method is where all the code
that draws the artwork goes and the
shape of a paint worklet is not
restrictive at all so you're free to use
all their class features to organize and
and do like whatever to organize your
drawing code any way that you prefer and
it's nifty because without a little
organization your drawing code can start
to look like this and we really don't
want that do we then moving on in the
same file after and outside of your
paintwork Lit class
you can call register paint which
requires two arguments the first is the
name the string name that the paintwork
should be registered under and then the
second this that allows it to be
accessed by the paint function and then
the second is the reference to the class
that you just wrote that does all of the
painting and when this all comes
together a paint worklet finally becomes
usable by CSS but how do we draw stuff
what does that even look like well the
API you use conveniently to draw stuff
is really just the canvas API is 2d
drawing context something that's been
around for a very long time that some of
us have good familiarity with and that
makes sense why why develop a separate
API for drawing in 2d when a perfectly
good one is like right there you know
the second argument is the canvas
geometry which offers the dimensions of
the drawing space and this is essential
to ensuring your artwork responds to the
available space because you know your
elements can kind of shift and change
size that sort of thing happens and in
this madly responsive world that we live
in we want to make sure that our artwork
adapts to the container that it's in so
here's an example of a minimally viable
paint method which draws a solid black
circle with a 64 pixel radius smack dab
in the middle of the canvas no matter
what the size of that canvas is and so
if we assume this super basic paint
worklet gets registered under the not at
all
ironic name of amazing and we apply it
to an element will get this this totally
amazing thing that in no way could be
replaced by a static image whatsoever so
this might be you right about now but
that simplistic example was meant to
illustrate how we get from one line of
CSS to something that lets you generate
background images using a familiar API
there is so much more that this
technology is capable of so let
take a look at it and just think what if
you could use this API to create
randomly generated artwork that could
enhance your designs and practical yet
spontaneous ways that are just a little
bit different a little bit of chaos
right and that's what we're going to do
we're going to talk about the other
important aspects of this technology as
well so what we're done here the only
thing that we'll be able to cage your
creativity is you so let's see what's
possible with this expressive and cool
technology but still and I sense a
little bit of hesitancy knowing where or
just how to start can be kind of
paralyzing so it's a good frame of mind
you know a good frame of mind to get
into is to think of the canvas is like a
set of tiles where all the tiles are
separate but potentially overlapping or
connected pieces which are randomly
generated this mindset trains your focus
conveniently only on what to do in each
tile rather than allowing yourself to be
overwhelmed by what to do with the
entire space and once everything is done
you can kind of step back and just see
how all these things connect and
interlock and how they come together and
this helps us reach where I think is the
sweet spot with the paint API we can get
weird yes but that weirdness can
intersect with what's practical and
therefore usable and that to me is like
the magical part of web development the
weirdly practical stuff now some of what
you're going to see here and I feel
obliged to call this out is very much
inspired by a wonderful talk that Tim
Holman gave at JFK FAU last year where
he talked about generative art using the
canvas API I highly recommend you check
it out on YouTube because he is way
better at this stuff than I am and
pretty much everything in that talk is
something that you will then be able to
apply to paint worklets in the CSS paint
API so do check it out okay let's
revisit that paint method in the amazing
paint wirklich class and just maybe
change things up a little bit there's
more going on in this example then when
we were just drawing this mundane lonely
circle in the middle of a bleak canvas
so first we establish the size of each
tile which will be 32 pixels
square then we divide the canvases width
and height by the tile size and then
we'll write two loops using those values
to draw each individual tile the first
loop deals with tiles on the y-axis and
then the second loop deals with
unsurprisingly tiles on the x-axis and
the second loop is nested inside of the
first one and this will end up
populating or allow us to populate every
tile available on the canvas with
whatever it is we feel like drawing so
what you do up here is up to you but in
this example I'm going to draw a line in
every tile but the random part a bit of
it will be where the line goes in in the
tile space right so in every iteration I
call the context objects begin path
method to begin drawing a line and then
I leave it to math dot random or math
dot pseudo random am i right to decide
if the line will cross the tile space
from the upper left corner to the lower
lower right corner or draw a line
straight line from the upper left corner
to the lower left corner and after that
I finish up Euler rate iteration by
telling the context object to draw the
strokes so what does that look like
turns out it looks like a kind of cool
thing that changes every single time and
that is the kind of spontaneity and
randomness I'm talking about here and
that's just what you can do if you only
know like just a teeny little bit of the
canvas API but with a little ingenuity
that little teeny bit of ingenuity can
go extremely far like what if we just
decided to throw in a little something
extra and randomly decide whether or not
to draw a dot in the upper left hand
corner of each tile this gives the final
artwork a sort of circuit board
appearance which i think is kind of neat
and we can also make the tile size a bit
bigger and we can change the color and
because a canvas background is
transparent by default we can specify a
CSS background color property to set to
set this artwork on whatever color it is
we want so I've kind of done like this
off black grayish thing now imagine this
pattern in a pages masthead background
but maybe at a more subtle and lower
opacity that just gives the design just
a little bit of extra oof and like chaos
but without overpowering the forward
and like making things like you know
just kind of subtle like you know being
cool so that was cool but let's do a
different take on this idea and let's
like let's experiment a little bit more
like what else can we do
so here's a paint work I call blotto
which draws circles of varying size and
opacity in every tile like kind of an
inkblot kind of thing and so there's a
bit of code here so let's just walk
through it at the very beginning a
beginning of blah toes paint method we
set some variables the tile size of
course 8 by 8 pixels and then we define
this amplitude variable which aids us in
sort of generating randomly sized
circles later on and then we calculate
the number of tiles on both axes and
then we store the result of pi times 2
which means that the circle that we draw
will be a full circle at 360 degrees and
then finally we set the fill style to
this sort of like nice little blue and
now here's where we do the heavy lifting
you'll note that the loops are the same
as with the circuit board pattern and we
we do all the work inside of the second
nested loop now where it changes is that
we generate a random alpha value by
getting the remainder of math dot random
divided against itself and this will get
us a valid transparency value from 0 to
1.0 and then we multiply the tile size
by math dot random and our amplitude
value from earlier and now all we do
from here is just draw a circle with
that information that we've just sort of
come up with and then we register good
ole blotto here like this just like
usual and then we can use it and CSS
like so and because paintwork let's draw
like I said they draw on a transparent
background by default we can add a nice
little background color to sit for it to
sit on I've chosen this sort of like
off-white and so after all of that
faffing about what does it look like
pretty cool I think it's not bad if I do
say so myself I mean I don't think
anything I do is cool but I think that's
kind of cool and as before we get a
unique result every time the canvas
redraws and it gives us again that
organic feel that while stable and
familiar is just slightly chaotic and
spontaneous and I really like that every
single time it draws ok so that was
alright but let's think outside of tiles
right tiles are fun in their convenient
ways to draw like generative artwork but
let's step into the world of
trigonometry and I'm sure most of you
are pretty comfortable with trig but if
not don't worry it's it's not too hard
and I never was a good math student so
like I was brushing up on my trig last
week
here's another paint worklet I call
flashy which randomly draw stylized rays
of light sorry
around a circle which flare outward
which makes it sort of look like a
stylized rendering of the Sun which i
think is kind of cool so let's look at
the paint method and as before we'll
need to establish variables that we can
use later the first is the radius of the
main circle that the Rays will radiate
from think like the Sun itself which
we've set at 48 pixels next is the
amount of deviation of each rays with
and this is what we use to make it look
like the Rays are flaring outward as
they extend from the Sun itself out to
the edge of the canvas and next is the
outer radius which is sort of think like
a giant circle around the entire canvas
that you don't see we do this so that we
ensure that the edges of the Rays will
be drawn outside of the canvas bounds
when we use our trigonometry functions
to plot where those rays need to go and
finally we have the X&Y coordinates from
which the circle the main circle will be
drawn and we calculate the position of
this thing by multiplying the width and
height of the canvases size by some
static values to sort of ensure it sits
proportionately somewhere in the upper
left but if the element sizes changes
it'll kind of adjust a little bit and
now it's time to get drawing so first we
set the fill color and then we draw the
circle from which the rays will radiate
pretty straightforward now here's where
it gets a little interesting and this
code will all be available so if you
can't totally read it that's ok it will
be available somewhere because we are
drawing rays
radiating around a circle we'll want to
do one for loop that starts at zero
degrees and then goes all the way around
to 360 and then in each iteration will
let math dot random decide if it wants
to draw anything at all and if it
decides okay we're going
do that we'll calculate the x and
y-coordinates at the edge of the larger
circle that exists outside the canvas
bounds and so then we do this by
converting the edge of each ray from
degrees represented by our incrementer
i2 radians and from there we'll then
draw this polyline shape that uses a
trigonometry like two trigonometry
functions to figure out the x and y
coordinates to draw the lines for each
ray once the shape is drawn we fill it
with the same solid color as we used
with the circle then like before we
register that flashy little fella and
then like before we use it in the CSS
like this and give it a nice pinkish
background color to sit on and what do
you think that looks like something
pretty cool I think right it looks kind
of neat and a little bit of mass goes a
long way into making this look really
cool in stylish and like before we still
get that spontaneity that gives it that
familiar but slightly different
appearance each and every time that it
draws so and as an aside I sucked at
math in school like like barely scraped
pie through algebra to sucked so if I
can make stuff like this pretty much
anyone can make stuff that looks really
cool with just a little bit of like you
know even just some basic arithmetic can
help you make some really cool stuff in
this and so a case in point like this
paint work that I like to call slapdash
this one is kind of one of the easier
ones to make and I use it on my website
currently although it might not be long
for this world I'm kind of getting tired
of it but that's okay like because I've
got some others up my sleeve like this
one which I like to call bumpy because
it's get it it's bumpy and then there's
this alternate rendering of the blotto
paint worklet which I showed you earlier
but with just sort of a larger radius on
the circles with a blend mode applied to
give it a sort of inkblot gone wild
effect that's kind of cool
and then there's yeah this blend mode
just kind of makes them like sort of
overlay each other I think it uses
multiply and then I started getting a
little bit whack here and came up with
this one which I like to call byte mare
and even though it looks 3d it's really
just a 2d drawing I don't think you can
use a 3d
context and paint worklets yet I use
some trigonometry functions just to kind
of fake that 3d look and I think it's
pretty convincing and in that same vein
here's my favorite one I made so far
which I call parallel oh wow it also
uses a little trig to fake that free D
thing again and I don't know who knows
where I'll go with with this next this
is just the last one I made that I like
but it's a lot of fun and if you're
looking to know how I did some of these
or where you can find that code I have a
little surprise for you at the end of
this talk but we got some other stuff to
talk about first okay so we had some fun
there for a minute
right like it was a wild ride but the
problem with paint worklets as I've
described them is that they're just not
well very flexible well as I described
them you just you have to change the
paint work look code itself to change
its appearance and while that can work
it's not very convenient and really just
kind of a pain what if you wanted to
reuse that same work lick code in
different places on the same site what
if you wanted to do wanted it to do
basically the same thing but adapt it to
different contexts solely by tweaking
its appearance and CSS that'd be cool
right guess what that is possible you
can set you can set up your paint
worklets so that their appearance can be
altered by what are called custom CSS
properties that you that you defined
that are relevant to your paint worklet
and this feature is not a part of the
CSS paint API but rather another feature
called the the custom properties or
maybe just properties and values API
seriously so let's go back to the CSS
for the flashy paint worklet example
from earlier
what would the CSS for this look like if
we could customize an aspect of its
appearance through a custom property
kind of something like this like if what
if we wanted to control the size of the
of the sun's radius right make that
circle bigger if we wanted to well the
the properties starting with double
dashes is a custom property and when we
set custom CSS properties on elements
using a paint worklet we can access
those values within that paint worklet
and now this is where the power and
practicality lies with this technology
because if a paint work
rendering can't be influenced by
anything outside of the worklet code
itself you'd have to change your worklet
code every freakin time you wanted it to
do something different or you'd have to
make multiple versions of that same
worklet in order to customize its
appearance in CSS and that's just
wasteful custom properties help us to
make paint worklets infinitely more
useful and reusable and they're like the
hook and a good song that like just in a
really good banging song that just makes
it even better so the next step is to
use a little thing called the properties
and values API to establish the identity
and shape of a custom property property
in your application JavaScript like you
would put this on the main thread in
your app code and even though it's not
strictly necessary to do this it's not a
bad idea for a few reasons for one you
can control what the data type is for
the custom property via the syntax
setting in this case you can see that
we're specifying the property's value as
an integer the value we set here is
called a CSS syntax string and it helps
the API to enforce what's considered a
valid value for a given custom property
and two we can decide whether or not the
value for this property can be inherited
by child elements using the inherits
option
I haven't found exactly where this is
useful yet for paint worklets but I'm
sure it's there somewhere I mean it's
it's specified I'm sure it's useful and
three we decide what the properties
default value should be via the initial
value option if that property is
unspecified so that's kind of useful
because properties have defaults that
just sort of kick in if you don't
specify them so we can do the same thing
with our custom properties so what does
it look like well you want paint
worklets to access those custom property
values it's not too ownerís really it's
just a two-step process the first is to
add a static getter method to your
worklet class called input properties
which returns an array of all the custom
properties that we've registered and
then in the worklet we can access these
properties through a third argument in
the paint method signature called
properties unsurprisingly the arguments
get method allows access
to the custom property values passed in
CSS here you can see that we like we
pull it and then we cast it to an int I
don't think the property properties and
values API is at candid candidate
recommendation so that might actually
not be the final code for what that
looks like
but then we get it in there and then at
this point we can then control the paint
worklets presentation entirely with CSS
and what we can achieve with this is
super duper powerful when we
parameterize paint worklets we make them
customizable in ways we didn't even
think was possible and here you can see
that we can dynamically change the tile
size of the blotto worklets output like
we can on we can change the color as
well as other influential properties
like the amplitude the maximum o path
opacity like the blend modes and stuff
it's pretty cool we can make them fit
practically anywhere and that's the
beauty of generative artwork applied in
this context it's one thing to generate
art with JavaScript we've been doing
that for a long time it's an entirely
different thing to control it in this
fashion and you can do it without having
to tweak more than a couple properties
so now that I've showed you all the cool
stuff it's time I inform you of the
unfortunate yet entirely predictable
reality that you must eat your
vegetables by which I mean it's time to
tell your paint to tell you that paint
worklets don't have solid browser
support and that therefore that means
you must treat them as a progressive
enhancement and you'll need a back-up
plan
right so that's a lot of what we do on
the web is feel like my official title
is like back-up plan
guy for everything furthermore because
paint worklets are JavaScript you need
that back-up plan
even in browsers that do support them so
support isn't terrible for the painting
API when you look at the bigger picture
in fact it's only one that's the only
part of Houdini that currently is at the
candidate recommendation stage at least
at the time of this tables rendering but
support for it just isn't universal so
you need to know how to adapt and that
said you shouldn't have to break your
back to provide a fallback because CSS
itself can usually solve this the thing
to remember about resiliency
when using the CSS pain API is that the
order in which we specify CSS properties
matters and that is what is going to
save you
going this route is usually enough for a
fallback browsers that understand this
API will use the second rule but those
that don't will fall back to the first
occurrence of that rule that they
consider to be valid and in this case a
browser like say Firefox which doesn't
support paint worklets yet but it's
coming will fall back to the first rule
and most of the time this approach is
enough the idea is that your fallback
should resemble the paintwork tlit like
someone right it doesn't usually have to
be much more than an approximation
because it's it's pretty rare that like
somebody would see both renderings of
like the fallback and the paintwork lid
itself like next to each other so to
illustrate this is my personal website
in chrome with the paint worklet used in
the backgrounds of different sections
it's just like that kind of slapdash
paintwork look that I had and now here
it is in a browser that doesn't and it's
not exactly the same but it's a
reasonable like kind of approximation
and it works well enough and just be
sure that you're optimizing your
fallback images that's the perf guy
within me just can't help but tell you
to do that if possible try to create
like tileable smaller patterns to keep
those fallback images as small as
possible because simple paint worklets
will usually just be smaller over the
wire than images now for another
question what if a paint work look just
like fails to load for whatever reason
and this is where promises are your
friend you might recall that in order to
load a paint work lift we have access to
a promise when it completely loads or
conversely when it fails to load and in
this case we can compensate for this in
the promises catch method which executes
if the promise was rejected in this case
if the paint work lead failed to load
for some reason and if it does fail we
can add a class the body element which
we can then target with CSS to provide a
fallback
that's the SS would look something like
this although that's a little weird
I said body element so work with me with
that inconsistency imagine it was
applied to the element itself instead in
the event that the paint work click
fails to load we would still get a stall
back background image that we can still
use and it's important to note that
least in chrome if a paint worklet fails
to load the CSS only fallback for non
supporting browsers described earlier
will not kick in because that it still
considers that paint syntax to be a
valid value but that worklet hasn't
loaded so you have to have a more
specific rule to make that work so
you'll need to provide an explicit
fallback otherwise you just you'll just
won't have a background at all okay most
of the talks I give are about
performance and I thought maybe I could
get through this one without any perf
talk but I was wrong it turns out that
there's a couple things to keep in mind
when it comes to keeping your paint work
looks fast as well as being considerate
with what with what can be a rather
taxing API if it's not used carefully
because if you don't you risk making the
internet more like this for people so
let's avoid that and talk about what you
can do to make paint work let's work as
optimally as they can because of their
restricted scope paint worklets won't
often be very large the ones I've
written usually end up being less than
two kilobytes before compression but if
you can help it do not run your
paintwork le'ts through Babel that is no
nose no so let's look at the impact as
an example that Babel transforms have by
taking the circuitry worklet from the
beginning of this talk even when it's uh
notified it's very small and it also
works in every browser that supports
paint worklets without babbles help
paint work 'let's do not need to be
transformed in order for them to work
now here's that same worklet after babel
transforms it and if you can't read this
code that's kind of the point there's a
lot of cruft here which affects parsing
compilation and execution time and the
omnipresent question of web development
seems to inevitably be should this work
in every browser and in this case there
are two answers no and hell no the Omni
from the paint worklets aren't supported
in all modern browsers yet so we don't
need to treat them with the web
compatibility mindset that we clung to
in the 2000s if you must must
if you must cut processed paint worklets
with babel perhaps out of a matter of
convenience and i can understand and
sure you know how to configure it to
minimize transforms I tend to avoid
including them in the asset graph of any
bundler until I know
be able to process them the way I expect
and this last piece of perfect base I
have is to be considerate of your users
particularly mobile ones
not everyone has a pixel 3 or a 20 18
macbook pro that can obliterate whatever
comes down the rendering pipeline
without turning into a fully functioning
panini grill
so when paint work let's draw in normal
scenarios such as unload or when the
orientation of the device changes they
can be resource intensive but not
ludicrously so but that does not mean
that even though worklets do their work
off the main thread like web workers
that they can't impact page
responsiveness this screenshot is from a
performance profiling session where a
browser was continuously resizing and a
paint worklet was redrawing every time
they worklets will redraw any time the
dimensions of their container change or
if CSS properties on that container
change so please resist the temptation
to like do stuff like make them redraw
constantly to like make them like
animate inefficiently or otherwise for
no real sensible reason and additionally
make a call on when it's not appropriate
to use these things we can't look at the
device's battery levels since the
battery status API is no longer a thing
but we do have another signal we can
look at and that is the device memory
API the amount of memory that the
approximate amount of memory that the
device has and as usual it's not
supported everywhere but checking for
support is as trivial as seeing if
navigator dot device memory is defined
and if it is we can get the course
amount of memory and here we've decided
that if a device has at least 4
gigabytes or more of memory we'll
register that paint worklet and then we
can or otherwise we can like do the
class hook where we decide to go to a
fallback and in any case if you're
concerned that any part of your paint
where click code is inefficient
anyway if you profile in chrome dev
tools in the performance profiler and
then after that go to the sources tab
you can see the impact of each line like
how much execution time and then you can
just decide whether or not that you want
to address any suboptimal behavior if
you can well it's that time but before I
properly part from you all I want to
call it a few resources I think are well
worth your time first off you know
Kravitz has been jamming on paint
worklets for a long time
and if you feel like my quick intro to
the technology just didn't cut it for
you I understand
then you Anna's article on the on the AP
I just cannot fail you she is just it's
superb at describing these things and
secondly Sam Richards did a more broadly
scoped talk on Houdini in general last
year at CSS conf a you some of what he's
talked about there may have changed a
bit since as Houdini is evolving but
it's still a great talk for anyone who
wants to know more about this awesome
set of API is that are coming down the
pike and now remember when I said there
was a surprise here it is if you're
interested in seeing an open source
showcase of the paintwork let's I've
written you can check out a site I've
made called paint lets its paint let's
go Roku app calm but I got a little tiny
URL thing there and I'll post the slide
deck later this will show each paint
worklet and it's custom properties so
you can experiment with each one and
they're free to download you can just
use them if you want I don't care if you
credit them I don't care what you do
with them and Plus each paint workflow
yeah I just said that so and if you have
a paintwork look that you've written
that you think is pretty cool I welcome
you to submit a pull request to the repo
or just show it to me and I can and I
can pull it in for you because I'm kind
of picky about how that gets done so if
you'd like want that cool let's let's
talk there's I'll post this you'll know
where this is all at pretty soon and of
course at the time I thought I was being
a clever little
when I wrote paint --let's but it
unsurprisingly turns out that someone
had the same idea and before I did and
that person was not surprisingly Yoona
Kravitz so if you want a different take
on the idea check out her extra dot CSS
site has some cool things that you can
check out and pull into your project as
well and with that I thank you for your
time your interest your everlasting love
of all things web which gives me no
shortage of hope and admiration for this
community and for welcoming me into your
amazingly beautiful and historically
awesome city here are my slides which
will be available at that URL hopefully
shortly if I can get Wi-Fi thing figured
out
and uh thank you all so very very much
[Applause]