How do Systems on a Chip Work? Understanding Microchips. By. Branch Education
Your smartphone has a variety of components that can do dozens of different things.
and the brain at the center of all of these components
and functions is the System on a Chip, or SoC for short.
When you watch a video such as this one, phone a friend across the country,
or navigate through a city using GPS, it may seem like magic, or alien tech,
but in reality these feats of technology are all performed inside this SoC
which is designed and manufactured through the hard work of
countless scientists and engineers. In this episode we’re going to explore each section
of this SoC and then piece together the processes
this system on a chip goes through when you do something
as routine as snap a photo and post it online.
These chips are unfathomably complex as there are between 5 and 10 billion transistors
that make up the SoC and they all need to fit into an area the size of a penny.
This is a complex topic, and there are many avenues that we could explore,
but in order not to overload your brain with details,
we’ve decided to focus on the following aspects.
First, we’ll start with a few notes and caveats, and then we’ll dive into the layout
and key sections of the system on a chip.
Next, we’ll explore how data flows through the SoC
using the example of what happens when you take a picture.
Then we’ll look at some of the complexity
inside the central processing unit or CPU block of the SoC.
And stick around until the end where we’ll discuss
how scientists and engineers design and manufacture these microchips,
and we’ll explore what a nanoscopic view of the SoC
would actually look like compared to this visualization.
So, for this video, grab a snack, sit down, and maybe take some notes if you feel like it
because this is going to be one very thorough and incredibly detailed video.
To start, this video is sponsored by Gerber Labs,
a quick, reliable, and user-friendly provider of printed circuit boards.
Next, this microchip is actually composed of two key parts.
On top is the DRAM or the dynamic random-access memory,
which is the working memory of your smartphone, and on the bottom is the SoC,
the brains of your smartphone.
FYI, the long-term memory, where all your music, apps,
files, and the operating system are saved, is over here in the flash storage.
We have a separate episode on that microchip, and you can find it here,
but in this episode we’re going to focus on the SoC.
This setup is called a Package on Package microchip or PoP
Technically this is the microchip, and the separate layers of the DRAM
and SoC are each called a die.
However, the word chip has evolved to mean a variety of things so,
while all of these rectangular components are microchips or chips,
both the DRAM and system on a chip are also called chips even though they're just
half of the overall package on package microchip.
Moving on, we may make comparisons or analogies
between smartphones and humans; for example we called the SoC the brain,
or the camera can be considered similar to a human eye-
however these are two very, very different systems,
and while the analogies may be useful conceptual tools,
the underpinning principles behind SoCs
and human brains and how they work are entirely different.
Finally, the companies that design these SoCs
are incredibly secretive about the specifics of how they work
and what the actual designs look like.
It takes dozens of different companies to design the optimal layout for
billions of transistors to perfectly execute the functions of a phone,
and a separate set of companies to manufacture the billions of chips used by all our phones.
So, in this video, we’ll provide you with the best information we can,
considering the companys’ secrecy of intellectual property.
As a result, this information will be mixed and woven together from different sources.
For example, this is an LGV10 smartphone from 2015
and it uses a Qualcomm Snapdragon 808,
but due to the limited availability of information on it,
we’ll have to use images and information from the
Apple A12 and A13 microchip in the IphoneX and iPhone 11,
as well as the Qualcomm snapdragon 865 in the Samsung S20.
All of these microchips have different performance metrics
and different capabilities, but fundamentally they work on similar principles.
Okay, with all these notes covered,
let’s move on to understand how this System on a Chip works.
A human brain has different sections for
processing information received by your eyes and ears,
a different section for speech, ones for controlling your taste,
smell, movement, balance, involuntary functions, and so on.
Rather similarly, this System on a Chip is divided into different functional areas.
Here is a graphical representation called a block diagram.
The key areas in the SoC are: the CPU containing multiple cores;
the GPU which renders graphics;
the shared memory cache which is usually around 4 to 8 megabytes;
the digital signal processor which interfaces with things like the speakers,
microphone, sensors and many other things;
the display engine which communicates with the touchscreen display;
the video processor which compresses and decompresses images and video
and enables 4k video recording and playback;
the image signal processor which processes pictures and video taken by the cameras;
the modem which interfaces with the various wireless networks;
the storage controller, which saves and loads information
from the flash storage microchip; the memory controller which connects
to the DRAM microchip located on top of this one;
the security enclave, which executes encryption,
and manages the public and private keys;
a number of peripheral functions such as clocks,
temperature sensors, debug ports, and general purpose inputs and outputs,
an always on microcontroller unit; and finally the network-on-chip or NoC
which arbitrates or manages data flow through the SoC,
the DRAM and other parts of the phone.
Furthermore, some SoCs may have on board power management circuits
that complement the power management performed by separate chips outside the SoC.
Additionally, some SoCs such as Apple’s,
have a neural processing unit or an NPU that is specialized circuitry
that can execute machine learning algorithms
far more efficiently than the CPU,
both in terms of speed, and more importantly, power consumption.
One confusing detail is that different generations of SoCs
use different marketing names and mix multiple functions into different functional blocks.
For example, here’s the block diagram of the snapdragon 808,
versus the diagram for the snapdragon 865
and here’s the block diagram for Apple’s A12 chip.
Each company’s marketing team may decide to provide differing block diagrams
to try to impress the consumers, which at times can certainly be confusing.
Let’s move on and explore a real-life example of how data moves
around your smartphone when you take a picture.
But before that, one quick note. If you enjoy watching this video,
or if you think it’s a useful learning tool, make sure to like and comment on this video
as well as share it with others. With your help, we can get this video to over
10,000 comments and more than 100,000 likes. But for now, let’s get back
to understanding what happens when you take a picture on your smartphone.
First, the photons from the scene enter the camera’s lenses,
flow through a color filter array, and hit the sensor’s photodiode pixels.
These color filtered photons are then absorbed by each photodiode
and converted into an analog electrical current
which is then converted into a digital 12-bit binary value.
A 12-megapixel camera’s image has 12 million of these 12-bit binary values.
Note, that each pixel is either Red, or Green, or Blue, and thus the overall image is considered raw
and still has to undergo a number of image processing steps
to turn it from a raw image into a recognizable picture,
where the pixels each have a red, and green, and blue value.
But before these processes take place, this image data must be stored somewhere,
and as a result it’s sent to the smartphone’s working memory or DRAM.
To do this, the data travels from the camera and enters the SoC
through a mobile industry processor interface or MIPI
which can send or receive data at around 5 to 8 Gigabits per second.
This image data is then routed by the Network on Chip arbitrator
through the SoC to the memory controller and into the DRAM,
which, as mentioned before, is located on the die above the SoC.
There are two quick things to note:
First, the data path between the SoC and the DRAM is shared by everything.
In this scenario it’s the job of the network arbitrator
to prioritize the incoming sensor data of the uncompressed raw image so that no data
from the camera is lost, and thus the arbitrator
streams the raw image directly into the DRAM.
Second, a 12-megapixel uncompressed raw image takes up around 24 megabytes
However, the memory cache on the SoC is typically 4 to 8 megabytes
and is shared among all the processes on the SoC.
Therefore, when the image signal processor or the video processor works on a
recently taken picture in order to make it viewable
or to compress it, they can only work on small subsections of the overall image,
leaving the entire image temporarily stored on the DRAM.
This also happens while watching a video,
playing a video game, or using pretty much any app.
Okay, let’s get back to the raw image data of the picture that was taken by the camera,
and then sent to the MIPI routed through the SoC, and stored in the DRAM.
The next step is for the image signal processor or ISP
on the SoC to read the raw uncompressed image data
and perform a number of image processing steps.
These steps involve first correcting for darker pixels
on the edge of the sensor due to lens shading.
Then the ISP performs the demosaicing process which involves taking the image data
and the pattern on the camera’s color filter,
and calculating a red, green, and blue value for every pixel.
Next the image signal processor denoises, sharpens, enhances the image,
and color corrects it because the red, green, and blue filters
on the camera don’t directly match the hue of the red, green, and blue pixels
in the display, and finally the image signal processor tone maps the image.
After all this we have a picture with 12 million pixels,
each with an 8-bit red, green and blue value.
To get the picture to promptly appear on your screen another series of steps are involved:
the RGB image data is taken from the image signal processor,
sent to the GPU where it gets overlaid into the graphics
of the camera app and scaled to fit the screen
The resulting RGB values get sent to the display processor,
and then the image is routed to the display where it’s converted into intensities of current
in order to light up the corresponding pattern of red, green and blue pixels.
And there you go- a picture taken by the camera and whoosh!
displayed on the screen.
But we’re not done yet. How is the picture saved on your smartphone?
Well, first the picture needs to be compressed,
and to do that the uncompressed image currently
residing in DRAM is sent to a dedicated video coding processor
where it first gets converted from red, green and blue values
into YUV, or luminance, blue chrominance, and red chrominance values.
Next, this data undergoes a series of algorithms in order to remove information
that is undetectable to the human eye and compresses the image
into approximately a 3-megabyte JPEG format.
This compressed image is sent back to the DRAM,
and then routed to the smartphone’s flash memory for long term storage.
Now, if you were to send this picture to a friend,
the compressed image would get brought back into the DRAM,
and then routed to the modem where it’s divided
and assembled into packets, then sent to the 4G, 5G or Wi-Fi microchip,
converted into electromagnetic waves and sent to a cellular or Wi-Fi network.
One thing we’ve talked a lot about is the movement of data through the SoC.
All of this data movement between the blocks runs
over connecting wires on what is part of the network on a chip.
This network has routers and switches for shared access
to routes and targets like the DRAM. The routers and switches arbitrate the flow of data
and act like digital traffic lights and the pathways act as a digital highway.
These highways have different widths of wires based on what they’re talking to and the
amount of information that needs to be sent. Depending on required data rates,
there are typically 128 or 256 wires running parallel to one another,
carrying one bit at a time, and operating between 500 and 1,500 Megahertz.
In order to avoid wasting power from your smartphone’s battery,
the frequency and resulting data transfer rates across
the network on a chip ramp up and down depending
on the requirements of the applications you’re running
and this feature is called dynamic frequency scaling.
Before we explore the CPU and see how these chips are designed and manufactured,
let’s take a quick break and talk about this video’s sponsor, Gerber Labs.
Every SoC eventually makes its way onto a
printed circuit board along with other components
that are put together to create a vast array of different circuits.
You can start your own creative circuits
by building a printed circuit board from Gerber Labs.
Gerber Labs is founded by electrical engineers for electrical engineers.
They make it easy to get boards
with consistent quality, quick turnaround, and reliable shipping.
Check them out with the link in the description
and stick to the end of this video to receive a discount code for your order.
Okay, so, thus far we’ve glossed over the
functionality of each of these blocks in the SoC,
but now let’s quickly take a look into the central processing unit or CPU section of the SoC.
As you see there are multiple cores,
and each core can run part of a program by executing instructions.
If we focus on just one of these cores, we see that inside it there is another incredibly
complex set of blocks that depict the different functional sections
and the data flow between them. There is a ton of information in this block diagram,
and we’re planning on making an entire series to explain it all,
but for now, here are the CPU’s memory caches.
The instructions such as add, multiply, load, store, compare, jump,
and many more, flow this way, and the actual data being processed flows this way.
And then over here are the blocks that actually
execute the arithmetic, branching, and storing of data.
Let’s move on to a few additional details about this processor.
All smartphones use a reduced instruction set computer,
or RISC architecture, and almost all of these architectures are licensed
from a company called ARM which stands for Advanced RISC Machines.
Some companies like Apple license the instruction set architecture or ISA
from ARM and use this ISA to design their processor cores
in house whereas other companies like Qualcomm license complete blueprints
which are technically called intellectual property cores from ARM.
Qualcomm then integrates the ARM IP cores into its SoCs,
sometimes keeping the design of the core as is,
but more often modifying or customizing the ARM core
to better suit Qualcomm’s design needs and,
in the process, Qualcomm rebrands the name of the CPU.
However, all this information still begs the question of,
how do these SoCs get designed and manufactured.
Instead of talking about the entire design process
which you can find in the creator’s comments,
we’re going to discuss two design principles of the SoC.
The first principle is called hardware acceleration,
and this principle deals with the fact that instead
of having a single very powerful general purpose CPU with a lot of cores,
this chip has a variety of special purpose blocks dedicated
to performing specific functions. These special function blocks,
or hardware accelerators, compute their tasks faster while using significantly less power
than if the same operation were performed by the general-purpose CPU.
For example, as we saw with the processes involved in taking a picture,
there is an entire block of the SoC that's dedicated
to processing the image, and then a separate block whose job it is to encode,
decode or compress or decompress the image,
and, if you’re watching this video on your smartphone,
it’s actively decompressing this video as you watch it.
These sections are useful for taking pictures, but critical for recording 4K video,
and saving battery life. Hardware accelerators are utilized for pretty
much all computationally-intensive tasks performed by your smartphone
The next design principle is that, because SoCs are used in smartphones,
which operate on a battery, there's a huge focus on having the SoC
consume as little power as possible. One example of low power consumption design
is in the CPU. Instead of having all high performance cores,
chip designers often follow a big-little design structure,
where there are 2 or 4 big cores that are
high performance cores but consume more power,
and 4 little cores that are lower performance but are energy efficient,
and your smartphone prioritizes using these lower
performance energy efficient cores when possible.
Furthermore, the design of the transistor, which is the smallest and fundamental
building block of the SoC, has had its design evolved to both be smaller,
but more importantly to consume less power. As of 2020 the latest technology of transistors
is called a gate all around field effect transistor.
Let’s finally move onto how these chips are manufactured.
To start, the lion’s share of all SoCs get manufactured by a single company called TSMC,
or Taiwan Semiconductor Manufacturing company,
and a smaller, yet increasing proportion of SoCs, are manufactured by Samsung.
These microchips are manufactured on 300mm
wide silicon wafers in factories called fabs. In order to manufacture an array of chips,
the silicon wafer has to go through a series of 120
to 160 processes or steps performed by dozens of different machines.
This is the most common sequence of steps for manufacturing microchips,
As you see, there are only a dozen or so steps,
but each of these processes are performed
dozens of times in order to build an incredibly complex layout of transistors.
But this video is getting too long, so we’ll have to save the details of each step
for a future video. For now, here’s what a nanoscopic view
of the SoC looks like. On the bottom is the silicon wafer,
and the billions of transistors are implanted and built into and on top of the silicon.
On top of the layer of transistors are layers
of local interconnects that connect transistors to one another,
and then on top of that are the global interconnects
that connect sections of transistors.
This is just a small fraction of the entire SoC layout.
Let's zoom out in order to get a sense of the size of these components.
I'm sure you've heard that transistors are incredibly small, and,
Well, here's a single grain of fine table salt. Mind-blowing- right?
It makes you think… the potential of humanity to reach incredible levels of
science and engineering is limitless.