[Music]
hi i'm carrie anne and welcome to crash
course computer science as we've
discussed throughout the series
computers have come a long way from
mechanical devices capable of maybe one
calculation per second to cpus running
at kilohertz megahertz speeds the device
you're watching this video on right now
is almost certainly running at gigahertz
speeds that's billions of instructions
executed every second which trust me is
a lot of computation in the early days
of electronic computing processes were
typically made faster by improving the
switching time of the transistors inside
the chip the ones that make up all the
logic gates alus and other stuff we've
talked about over the past few episodes
but just making transistors faster and
more efficient only went so far so
processor designers have developed
various techniques to boost performance
allowing not only simple instructions to
run fast but also performing much more
sophisticated operations
[Music]
last episode we created a small program
for our cpu that allowed us to divide
two numbers we did this by doing many
subtractions in a row so for example 16
divided by 4 could be broken down into
the smaller problem of 16 minus four
minus four minus four minus four when we
hit zero or a negative number we knew
that we were done but this approach
gobbles up a lot of clock cycles and
isn't particularly efficient so most
computer processors today have divided
as one of the instructions that the aou
can perform in hardware of course this
extra circuitry makes the alu bigger and
more complicated to design but also more
capable it's a complexity for speed
trade-off that has been made many times
in computing history for instance modern
computer processors now have special
circuits for things like graphics
operations decoding compressed video and
encrypting files all of which are
operations that would take many many
many clock cycles to perform with
standard operations you may even have
heard of processors with mmx 3d now or
sse these are processors with additional
fancy circuits that allow them to
execute additional fancy instructions
for things like gaming and encryption
these extensions to the instruction set
have grown and grown over time and once
people have written programs to take
advantage of them it's hard to remove
them so instruction sets tend to keep
getting larger and larger keeping all
the old op codes around for backwards
compatibility the intel 4004 the first
truly integrated cpu had 46 instructions
which was enough to build a fully
functional computer but a modern
computer processor has thousands of
different instructions which utilizes
all sorts of clever and complex internal
circuitry now higher clock speeds and
fancy instruction sets tend to lead to
another problem getting data in and out
of the cpu quickly enough it's like
having a powerful steam locomotive but
no way to shoveling coal fast enough in
this case the bottleneck is ram ram is
typically a memory module that lies
outside the cpu this means that data has
to be transmitted to and from ram along
sets of data wires called a bus this bus
might only be a few centimeters long and
remember those electrical signals are
traveling near the speed of light but
when you're operating at gigahertz
speeds that's billionths of a second
even this small delay starts to become
problematic it also takes time for ram
itself to look up the address retrieve
the data and configure itself for output
so a load from ram instruction might
take dozens of clock cycles to complete
and during this time the processor is
just sitting there idly waiting for the
data one solution is to put a little
piece of ram right on the cpu called a
cache there isn't a lot of space on a
processor's chip so most caches are just
kilobytes or maybe megabytes in size
whereas ram is usually gigabytes having
a cache speeds things up in a clever way
when the cpu requests a memory location
from ram the ram can transmit not just
one single value but a whole block of
data this takes only a little bit more
time but it allows this data block to be
saved into the cache this tends to be
really useful because computer data is
often arranged and processed
sequentially for example let's say the
processor is totaling up daily sales for
a restaurant it starts by fetching the
first transaction from ram at memory
location 100. the ram instead of sending
back just that one value sends a block
of data from memory location 100 through
200 which are then all copied into the
cache now when the processor requests
the next transaction to add to its
running total the value at address 101
the cache will say oh i've already got
that value right here so i can give it
to you right away and there's no need to
go all the way to ram because the cache
is so close to the processor it can
typically provide the data in a single
clock cycle no waiting required this
speeds things up tremendously over
having to go back and forth to ram every
single time when data requested in ram
is already stored in the cache like this
it's called a cache hit and if the data
requested isn't in the cache so you have
to go to ram it's called a cache miss
the cache can also be used like a
scratch space storing intermediate
values when performing a longer or more
complicated calculation continuing our
restaurant example let's say the
processor has finished totaling up all
of the sales for the day and wants to
store the result in memory address 150.
like before instead of going back all
the way to ram to save that value it can
be stored in cash copy which is faster
to save to and also faster to access
later if more calculations are needed
but this introduces an interesting
problem the cache's copy of data is now
different to the real version stored in
ram this mismatch has to be recorded so
that at some point everything can get
synced up for this purpose the cache has
a special flag for each block of memory
it stores called the dirty bit which
might just be the best term computer
scientists have ever invented most often
this synchronization happens when the
cache is full but a new block of memory
is being requested by the processor
before the cache raises the old block to
free up space it checks its dirty bit
and if it's dirty the old block of data
is written back to ram before loading in
the new block another trick to boost cpu
performance is called instruction
pipelining imagine you have to wash an
entire hotel's worth of sheets but
you've only got one washing machine and
one dryer one option is to do it all
sequentially put a batch of sheets in
the washer and wait 30 minutes for it to
finish then take the wet sheets out put
them in the dryer and wait another 30
minutes for that to finish this allows
you to do one batch of sheets every hour
side note if you have a dryer that can
dry a load of laundry in 30 minutes
please tell me the brand model in the
comments because i'm living with 90
minute dry times minimum but even with
this magic clothes dryer you can speed
things up even more if you parallelize
your operation as before you start off
putting one batch of sheets in the
washer you wait 30 minutes for it to
finish then you take the wet sheets out
and put them in the dryer but this time
instead of just waiting 30 minutes for
the dryer to finish you simultaneously
start another load in the washing
machine now you've got both machines
going at once wait 30 minutes and one
batch is now done one batch is half done
and another is ready to go in this
effectively doubles your throughput
processor designs can apply the same
idea in episode 7 our example processor
performed the fetch decode execute cycle
sequentially and in a continuous loop
fetch decode execute fetch decode
execute fetch decode execute and so on
this meant our design required three
clock cycles to execute one instruction
but each of these stages uses a
different part of the cpu meaning there
is an opportunity to parallelize while
one instruction is getting executed the
next instruction could be getting
decoded and the instruction beyond that
fetched from memory all of these
separate processes can overlap so that
all parts of the cpu are active at any
given time in this pipeline design an
instruction is executed every single
clock cycle which triples the throughput
but just like with caching this can lead
to some tricky problems a big hazard is
a dependency in the instructions for
example you might fetch something that
the currently executing instruction is
just about to modify which means you'll
end up with the old value in the
pipeline to compensate for this
pipelined processors have to look ahead
for data dependencies and if necessary
store their pipelines to avoid problems
high-end processors like those found in
laptops and smartphones go one step
further and can dynamically reorder
instructions with dependencies in order
to minimize stores and keep the pipeline
moving which is called out-of-order
execution as you might imagine the
circuits that figure this all out are
incredibly complicated nonetheless
pipelining is tremendously effective and
almost all processes implement it today
another big hazard are conditional jump
instructions we talked about one example
a jump negative last episode these
instructions changed the execution flow
of a program depending on a value a
simple pipelined processor will perform
a long stall when it sees a jump
instruction waiting for the value to be
finalized only once the jump outcome is
known does the processor start refilling
its pipeline but this can produce long
delays so high-end processors have some
tricks to deal with this problem too
imagine an upcoming jump instruction as
a fork in the road a branch advanced
cpus guess which way they're going to go
and start filling their pipeline with
instructions based off that guess a
technique called speculative execution
when the jump instruction is finally
resolved if the cpu guess correctly then
the pipeline is already full of the
correct instructions and it can motor
along without delay however if the cpu
guessed wrong it has to discard all of
its speculative results and perform a
pipeline flush sort of like when you
miss a turn and have to do a u-turn to
get back on roux and stop your gps is
insist on shouting to minimize the
effects of these flushes cpu
manufacturers have developed
sophisticated ways to guess which way
branches will go called branch
prediction instead of being a 50 50
guess today's processes can often guess
with over 90 accuracy in an ideal case
pipelining lets you complete one
instruction every single clock cycle but
then super scalar processors came along
which could execute more than one
instruction per clock cycle during the
execute phase even in a pipeline design
whole areas of the processor might be
totally idle for example while executing
an instruction that fetches a value from
memory the alu is just going to be
sitting there not doing a thing so why
not fashion decode several instructions
at once and whenever possible execute
instructions that require different
parts of the cpu all at the same time
but we can take this one step further
and add duplicate circuitry for popular
instructions for example many processors
will have four eight or more identical
alus so they can execute many
mathematical instructions all in
parallel okay the techniques we've
discussed so far primarily optimize the
execution throughput of a single stream
of instructions but another way to
increase performance is to run several
streams of instructions at once with
multi-core processors you might have
heard of dual core or quad-core
processors this means that there are
multiple independent processing units
inside of a single cpu chip in many ways
this is very much like having multiple
separate cpus but because they're
tightly integrated they can share some
resources like cache allowing the cores
to work together on shared computations
but when more cores just isn't enough
you can build computers with multiple
independent cpus high-end computers like
the server streaming this video from
youtube's data center often need the
extra horsepower to keep it silky smooth
for the hundreds of people watching
simultaneously two and four processor
configurations are the most common right
now but every now and again even that
much processing power isn't enough so we
humans get extra ambitious and build
ourselves a super computer
if you're looking to do some really
monster calculations like simulating the
formation of the universe you're going
to need some pretty serious computing
power a few extra processes in a desktop
computer just isn't going to cut it
you're going to need a lot of processors
no no even more than that a lot more
when this video was made the world's
fastest computer was located in the
national supercomputing center in wushi
china the sunway tauhau light contains a
brain melting 40 960 cpus each with 256
cores that's over 10 million cores in
total and each one of those cores runs
at 1.45 gigahertz in total this machine
can process 93 quadrillion that's 93
million billions floating point map
operations per second known as flops and
trust me that's a lot of flops no word
on whether it can run crisis at max
settings but i suspect it might so long
story short not only have computer
processors gotten a lot faster over the
years but also a lot more sophisticated
employing all sorts of clever tricks to
squeeze out more and more computation
per clock cycle our job is to wield that
incredible processing power to do cool
and useful things that's the essence of
programming which we'll start discussing
next episode see you next week
crash course computer science is
produced in association with pbs digital
studios at their channel you can check
out a playlist of shows like pbs idea
channel physics girl and it's okay to be
smart this episode was filmed at the
chad and stacey evergult studio in
indianapolis indiana and it was made
with the help of all these nice people
and our wonderful graphics team thought
cafe thanks for watching i'll see you
later
[Music]
you