everyone
our story begins with a humble await
fetch these two lines of code make an
HTTP request wait for a response and
print out the status code while this
code is waiting a cascade of activity is
taking place in your JavaScript runtime
your operating system and your hardware
in this talk we're going to look at what
happens between these two lines of code
and what waiting can teach us about
software design now you may be thinking
hold up why do I need to think about the
stuff while writing JavaScript code and
more so a hundred feet from a beautiful
beach well I can sympathize I've
wrestled with motivating myself to dig
into these lower level details before
and I'd like to offer two motivating
ideas may be enough to get you over the
post-lunch home first
this isn't trivia these are real systems
we use every day while writing
JavaScript code understanding systems
teaches us examples of how problems can
be solved and most importantly it gives
us the opportunity to identify patterns
that guide how new systems can be built
second computers are really complex and
powerful and that can sometimes make
them feel magical but much of that
complexity comes from layer upon layer
of simple solutions to simple problems
as JavaScript programmers these problems
are relatable to us because we live in
an asynchronous world by understanding
the problems the lower layers solve we
can gain a greater confidence in the
face of this complexity as you'll see a
lot of that complexity serves a simple
task how computers wait my name's max
I'm visiting from San Francisco and I'm
a developer experience engineer at
patreon on the first Wednesday of every
month I co organize a local meetup
called waffle Jas you can see the logo
on screen it's a really fun time I work
with a team of some amazingly talented
people who I've learned a ton from and
if you're ever passing through I invite
you to come join us it's a really fun
night or come give a talk to talk to me
after the after this so back to waiting
waiting is kind of a strange concept and
it was difficult for me to like nail
down in writing this talk because we
rarely do it we spend as little time
waiting as humanly possible if I ask you
to wait for something you're gonna
switch to something else to do so our
use of the word wait is usually in the
context of a set of instructions to do
something and when there's some
dependency in that process we have to
wait for
our progress into the pillow progress is
no longer blocked by the way thanks
Andres for the cute spaghetti image on
this slide good recipes take advantage
of the idle time by interleaving
processes while you're blocked on one
thing you can start on another this is a
great analogy to how computers multitask
except computers have thousands of
different processes they're carrying out
at any given time and unlike most
recipes the order in which things can
happen is unpredictable as programmers
we like to think in terms of sequential
steps because it makes our codes simple
to reason about you make an HTTP request
then you wait for the response then you
print out the status this is why async
await style programming is so useful it
lets us express steps in a clear and
linear order waiting makes that possible
it's the glue that allows us to express
a series of separate events linearly but
when we say to wait we expect that the
human or computer on the other side is
going to find something else to do in
the meantime this is a behavior we have
to design into our computers let's dive
into the mechanisms of how this works
starting with a simple microprocessor
for simplicity I'm going to focus on a
single core CPU that means it can only
do one thing at a time
Rich's talk which is immediately after
this at around 1:30 is going to cover
threading which takes advantage of
modern CPUs having multiple concurrent
course so back to the CPU we're gonna
start with a simple sub component called
the clock the clock periodically pulses
an electrical signal on and off and this
is what it drives instruction execution
in your CPU in modern CPUs we measure
the frequency of the clock in gigahertz
meaning the clock is oscillating
billions of times per second from a
reductionist point of view every single
thing a computer does or waits for
begins at one clock cycle and ends with
another by programming a microprocessor
we can implement a simple kind of
waiting called busy waiting this is a
loop where each cycle we check the
condition we're waiting for if you want
to do other things while we wait though
we kind of have to intersperse them in
this wait loop and that increases the
time between checks as a number of
things we want to wait for increases
this loop becomes less and less
efficient because you're checking more
things each type you cycle that's where
interrupts come in interrupt signal the
CPU when something happens such as your
network interface receiving data a
keyboard key being pressed or timer
elapsing
when a processor receives an interrupt
it pauses what's currently running saves
the execution State and switches to
different code called an interrupt
handler the interrupt handler takes any
immediate actions necessary and then
determines how to proceed forward the
code that implements interrupt handlers
is part of your operating system the OS
makes it possible to write higher-level
programs that don't have to worry about
interrupt handling and communication
with hardware the OS governs switching
between which programs are running so
that multiple programs can take turns
sharing a CPU this is called scheduling
the operating system also provides api's
for i/o called system calls for things
like writing the files and sending
packets in Linux most kinds of i/o is
represented as operations on streams of
bytes here's a couple examples on-screen
IO takes time though you know disks take
time to perform operations network
devices take time to transfer data when
a program is performing i/o it often
wants to wait until that IO is completed
so a simple model for this is called
blocking i/o while the program is
waiting for an i/o operation we say that
the calling program blocks until that IO
completes once the OS receives an
interrupt that the AI is completed the
process is queued to be resumed by the
scheduler while that process is blocked
the OS scheduler can run something else
in the meantime
simple blocking i/o system calls only
wait for one thing at a time though if
we want to wait for multiple things
we're gonna need some more tools the
operating systems provide non blocking
versions of many i/o calls for this when
you open a file or network connection
the OS returns a number that identifies
the stream called a file descriptor you
can use that FD to reference the stream
and other system calls so in this
example if an operation would block the
non blocking read returns an error
instead if we get an error indicating
the read would block that means there's
no late data left in the buffer for us
to read so instead we can find something
else to do instead of pausing execution
this leaves our process running so we
get to decide what to do instead of
blocking non-blocking i/o calls can also
be used to wait for multiple things at a
time for example we can loop over a set
of file descriptors and try to read from
each of them if there's no data to read
we continue on to the next file
descriptor
however now we're back to essentially
busy waiting what we really want to do
is block on a set of things that could
happen resuming when any one of them
does operating systems provide event
based system call
for this a simple one is called select
select is given three sets of file
descriptors one for each kind of event
stream is ready to read streams ready to
write and streams with errors the Select
call then blocks until an event happens
or a specified amount of time elapses it
then returns to the programmer which
file descriptors can be read to written
to or have errors so here's a really
simplified example of how select works
we pass a set of file descriptors we're
interested in reading from and then it
blocks until one or many of them becomes
readable when select returns it gives us
the list of files that now have data
available and we can loop over them and
read from them without blocking each
operating system provides a slightly
different modern implementation of a
venture of an i/o though linux this is
called eople mac OS and bsd fkq and
Windows has iocp to write cross-platform
software we have to implement different
i/o code for each one some programs do
that but many others use libraries that
abstract over the differences between
these api's this is where libuv comes in
Lybia v abstract so ver the varying
implementations of event-driven i/o in
different operating systems and provides
a common interface for them Libby V is
used by node which we'll be using as our
example JavaScript runtime in the future
of these slides I'm guessing there may
be a few Lybia V developers in the
audience today if you can find one of
them give them a high-five for me okay
Libby Vee lets you perform network and
file operations and register callbacks
and events happen Libby Vee uses the
operating systems event-driven i/o
constructs to wait for events and when
something happens
Libby P then executes the registered
callbacks and resumes waiting this is
called an event loop and here's an
example to make a network connection
we initialize what libuv calls a handle
which represents some kind of i/o object
we can perform operations on when we
open the network connection we pass
Libby via callback to run when the
connection is established this is very
familiar if you're used to like running
JavaScript code and assigning callbacks
when things complete to read data from
the connection we need to tell we BV to
track when the connection becomes
readable we provide a read callback
which will be called with the data as it
becomes available and then the last
thing we need to do is run the bv event
loop this is going to block until the
next IO event we're interested in happen
and call the related callback for us no
js' is implemented on top of this Libya
via vent loop and when J s uses a node
API to perform IO under the hood notice
calling libuv to perform it and then
when IO events happen node runs the
callbacks or triggers events in
JavaScript so that was a lot now that we
have our cast of characters let's return
to our code in order to make an HTTP
request we have to perform several
network operations including looking up
the host name opening a socket sending
an HTTP request and receiving data I'm
going to gloss over many of those steps
so we can just look at a single walk
through the layers as we establish a
socket connection so let's dive in and
break this down let's start with the
fetch call fetch uses nodes HTTP module
to start the request it then returns a
promise that represents the pending
value of the fetch response our
JavaScript awaits the response from the
fetch this tells node to pause and save
the JavaScript state and then switch to
running up other queued up JavaScript
code we're gonna skip over looking up
the IP address here and go straight to
opening a connection so node uses libuv
to open a connection to jcsoh I calm and
accuse up a callback to run when the
connections established to accomplish
this libuv tells linux to make the
connection at this point if node has no
more JavaScript to run it then calls
into Libby V's event loop which will
wait for the next i/o event so now no
js' is waiting under the hood libuv is
using the Linux pull API to track these
i/o events libuv tells Linux it's
interested in when the socket becomes
writable then libuv waits as well
locking until a timeout or the next
event happens in the meantime Linux is
Network stack is busy making the
connection this involves the operating
system Network driver sending data and
the CPU transferring that data to the
network device while we wait for a
response the operating system is going
to switch to running other things
eventually though the CPU will receive
an interrupt that we're interested in
that the network device has received
data this interrupt will cause the
operating system Network driver to read
the data from the network interface
and then this is going to continue back
and forth for a while until the
connection is totally established at
this point the connection socket becomes
ready to write which is what libuv is
waiting for Libya V execute any
callbacks waiting on file descriptors
that became ready and it just so happens
that we have one from node waiting for
the connection to be established once
we've finished processing all waiting
callbacks we finished our first
iteration of the Libya v event loop
there's going to be like a couple other
things that happen here that I'm
glossing over there's several similar
back and forth that we've seen like
starting a secure TLS connection
handshake and actually making the HTTP
request we're gonna go out skip over
them here but when that's all finished
nodes HTTP module is going to emit a
response event this is what our fetch
promise that we initially made is
waiting for when the promise resolves
the wait is now ready to resume so node
execute our next line of JavaScript code
printing out the response code that was
a lot to digest huh let's look at the
key parts in one picture so your
JavaScript code awaits a promise no js'
accomplishes this by pausing execution
until the promise resolves under the
hood node lit uses libuv
to queue up network operations Libby Vee
uses the Linux Ipoh API to listen for
events when the network connection is
ready so I told you I was gonna explain
how to wait but like we're in this
process are we actually waiting and how
does the waiting happen let's break this
down a little bit more the first three
steps are essentially creating
information when the request finishes
resumed my JavaScript code when the
connection is established run my libuv
callback we're not waiting here we're
really just defining relationships
between what we're waiting for and what
to do when it's ready in contrast the
operating system in CPU are responding
to real world events and executing the
code that's queued up waiting for them
let's recap and summarize what's
happening when our JavaScript code waits
together our JavaScript code nodejs and
Libby Vee are performing i/o and queuing
up handling the responses when this iowa
event happens run this javascript code
the OS juggles incoming i/o events and
runs processes as soon as they become
unblocked
the CPU is busy executing these
instructions and then it gets
interrupted as real-world events happen
intuitively this is similar to how
humans wait we keep track of something
we're waiting for and do things that
were not blocked on until something
interrupts us and there's something new
to do
so far we've thought of CPUs in terms of
being a black box that just does i/o for
us but modern CPUs are really
interesting they can actually perform
many async operations in themselves that
will be pretty familiar to you as a
JavaScript programmer modern CPUs
offload data transfers using something
called DMA direct memory access this is
a subsystem that transfers data between
devices and memory in the background
when a transfer finishes the CPU gets
notified B an interrupt think of it like
a callback in JavaScript the CPU makes a
DMA request it transfers in the
background and then later on the CPU is
notified when it completes CPUs can also
set timers to trigger in an interrupt
and a specified time using the high
precision event timer this is really
analogous to set timeout and set
interval in JavaScript the CPU can set
background triggers to fire after some
time has passed so as you can see like
as JavaScript programmers we're really
familiar with asynchronous callbacks and
set timeout and set interval the CPU can
do all of this so when work is happening
in the background
sometimes the CPU actually doesn't have
anything to do there's no interrupt
instruction that needs to be run before
the next interrupt so how does the CPU
wait the first thing CPUs can do is
lower their clock speed this reduces the
amount of power the CPU consumes and the
amount of heat it generates at the cost
of slower execution the second thing
CPUs can do is progressively turn
themselves off in a modern until our AMD
CPU a core can issue a halt instruction
this will cause the CPU to stop its
executing instructions until an
interrupt is received this allows the
CPU to power parts of itself off
resulting in a significant energy
savings as an aside this is one reason
why requestanimationframe is useful
requestanimationframe is used by
front-end JavaScript code to schedule
animation timers as you can probably see
I think animations are awesome
personally but to save battery we should
use them sparingly requestanimationframe
can reduce the frequency of animations
or skip them entirely to save battery
which enables 2cp
enable CPUs to enter powered off sleep
states
if we look at how weighting is
implemented at each layer of the stack
what we're trying to accomplish and how
we accomplish it bears a resemblance
each of node libby v and the OS
scheduler consume incoming notifications
that something happened they then
dispatch handlers to respond to each
incoming event when there's nothing left
to do the loop sleeps until new events
are available in a nutshell we're
running in a loop we consume a queue of
incoming events and then wait until more
ready for us this is where the named
event loop comes from so for example the
nodejs event loop is consuming libuv
callbacks it runs a queue of JavaScript
callbacks and then it pauses and blocks
the process essentially to wait for the
next limit V callback the contrasts the
Linux scheduler it's it's it's not
exactly the same because Linux
schedulers actually always executing
something and it's getting interrupted
but those interrupts come in they get
stored and queued up and then the Linux
scheduler runs a Q the Q of unblock
processes and then it'll continue on its
way until the next interrupt comes along
and maybe if there's absolutely nothing
to do the Linux scheduler will issue a
halt instruction which will turn off the
CPU or turn off parts of it if you
squint both the operating system and no
GS appear to be doing kind of similar
things here the operating system enables
programmers to write linear blocking i/o
whereas nodejs enables developers to
write sequences of i/o using callbacks
or a sync await code both the operating
system scheduler and nodejs is event
loop allow multiple sequences of code to
be interleaved one big difference
between these two systems is that the
operating system implements pre-emptive
multitasking if a CPU interrupt occurs
while a process is running the OS make
pause the process and switch to another
in addition the operating system
scheduler uses a timer so it can
regularly interrupt processes so that
each gets its phone share fair share of
the CPU time in contrast JavaScript
runtimes interrupt implement cooperative
multitasking this means that the runtime
will not interrupt any running
JavaScript code the event loop only runs
in the spaces between JavaScript
execution your JS can and will delay the
event loop from pan
events this is actually a deliberate
design choice and it provides some
advantages to JavaScript programmers
while javascript code is running
incoming i/o can't interrupt or change
the state of the JavaScript world this
also gives JavaScript code the ability
to choose exactly when it wants to yield
to the event loop then why do we need
the OS layer at all wouldn't our code be
more efficient without it potentially
you could do that but what you lose is
generalism our computers are typically
doing much more than just running our
JavaScript code and not all software is
written in JavaScript however it is
possible to eliminate the OS layer
entirely and build runtimes that operate
directly on the CPU and hardware these
are called Yuna kernels some are
research projects some are more actually
used in production in general though we
want our computers to be able to run
many different kinds of software at once
and that's where an extra layer of
operating system comes in handy what if
instead we eliminated the event loop and
just let the operating system schedule
blocking i/o why do we need the event
loop at all can we just run a lot of
processes the answer lies in economies
of scale each operating system process
has overhead processes take up memory it
also takes time to switch between them
while we save and restore the execution
state by using a venture of an i/o and
an event loop a single process can
manage a vast number of async IO
operations in bulk overhead is lower
because the single process can use more
compact data structures for keeping
track of state the event loop can also
exert finer grained control over the
priority in which it handles different
events compared to the operating systems
more generic scheduler the operating
system knows much less about what we're
doing but this is a JavaScript talk and
what does this all have to do with
JavaScript and what does it have to do
with waiting
well most interactive applications can
actually be thought of as a tapestry of
event-driven code we weave together a
sparse graph of code that runs in
response to external I o events
happening and in between those short
bursts of activity our code isn't doing
very much our code is actually just
waiting a great abstraction layer breaks
down a complex problem into multiple
similar pieces blocking cylla o enables
us to think about this tapestry as
linear flows of events
this is what makes no js' and
async/await such a powerful tools they
give us leverage over complexity under
the hood the event loop ingests an
unpredictable stream of incoming events
and advances the linear flows we've
defined great abstractions also tend to
make both sides of their problem space
more efficient blocking silo not only
helps programmers understand their data
flows it makes explicit what processes
are waiting for so that our runtime can
focus its attention elsewhere and yet
abstractions always leak what we do in
our JavaScript code has impacts across
the entire system even though sometimes
we have limited control over it as our
stack of abstractions gets deeper the
distance between what we intend to
happen and how it happens widens before
we wrap up I'd like to point out to
newish layers of the stack we haven't
really gotten to yet which are pretty
interesting ones on the backend and
another's on the front end if your
back-end code is running in the cloud
it's not running on a real CPU there's
an additional virtualization layer which
provides a virtual CPU to our operating
system called a hypervisor the
hypervisor has its own scheduler which
allocates CPU and i/o time based on how
much you're paying for so you're
actually running an even more schedulers
than you thought however if your
front-end code is running on react 16 or
later your Dom renders are subject to an
additional scheduler - called react
fiber because the event loop doesn't
have a way to preempt control from
JavaScript when it's taking too long
react fiber schedules itself it's saves
State and pauses when it's out of time
and resumes after the browser renders
the next frame the fiber scheduler
splits up large UI rear Enders into
smaller pieces of work that can execute
incremental e it also prioritizes
time-sensitive work like updating input
elements so here's like a broad overview
of the many layers of our stack when
we're working both in back in JavaScript
and in front-end JavaScript there's a
lot of layers here and there's a
tendency for us to think of high level
programming languages like JavaScript as
less efficient because they introduce
additional layers of overhead and give
us less control over the execution I
want to leave you with more nuanced idea
than that as we've seen with waiting
working at a high level of abstraction
often leaves you with a clearer
expression
of what you're writing the more clearly
programmers can express their intentions
the more information we can weave into
our tapestry of events and the more
latitude we provide our execution layers
to optimize them the real cost of adding
too many layers is mismatching
abstractions where we waste energy
translating one problem into another as
you've seen many layers of modern
computers provide means of waiting in
similar ways over time we're gonna
simplify away some of those layers and
specialize them to better serve the
needs of modern applications at the same
time we can expect that the future will
demand even higher level languages or
high-level abstractions built in those
languages like react as you've seen
JavaScript gives us tremendous leverage
over the CPU and OS to weave together
flows in a way that's intuitive to us as
programmers but programming is still a
young medium and we still have much
learn about how to structure our ideas
if you're interested in diving in deeper
on these topics
I'd recommend three awesome resources
Sam Roberts is talk on how up close
shows up close how Libby V is used to
create the Jas event loop and exposes
the trade-offs that are made in
designing it Cindy sweethearts talk on
non-blocking i/o dives deeper into how
the operating system implements
event-driven IO internally and finally
Merrick Mitch cows Keast blog series on
select walks through the history of how
i/o multiplexing evolved starting in
early computer systems in a time before
people are just figuring out in a time
when people were just figuring out how
to write networked computers like at the
dawn of the internet it's really
fascinating stuff and I highly recommend
diving in and with that I'm gonna yield
back to the conference loop thanks for
scheduling your attention for the past
25 minutes
you