So.
If two kids in a trench coat can pass for an adult, then why can't we do the same with
two chips?
With the end of Moore's Law, people have increasingly turned to advanced packaging technologies
as a way to keep grinding out performance gains.
Die stacking is at the heart of this next-generation trend.
In this video, I want to talk about stacking dies using modern 2.5D and 3D integration
technologies.
## 2.5D and 3D
We need to start with definitions first because the names
and terminologies can get pretty confusing.
The packaging industry is full of jargon and it drives me nuts.
2.5D and 3D packaging are subclasses of what we call "Multichip Package" or "System-in-Package"
solutions.
This means that there are multiple dies or packaged devices inside a single package.
There is no formal definition for what exactly constitutes 2.5D or 3D.
I mean, who would be policing it?
But in general, 2.5D integration co-packages dissimilar parts - logic, memory, or whatever
have you - side-by-side on top of a connective substrate.
3D integration on the other hand co-packages dissimilar parts in a vertical stack, connected
together using a variety of technologies.
## Why the D?
Another question we should discuss is why do we need all these advanced packaging technologies?
As we all know, Moore's Law is dead.
We can still cram more components onto the same piece of real estate, but the costs of
doing so are onerous for all but the richest of customers.
But there are other ways to eek out more performance out of our systems.
The industry has coalesced towards two general directions to do it.
The first would be to add accelerator chips next to the central processing units.
These accelerators would specifically run certain algorithms.
The second would be to optimize cost by partitioning out certain parts of a monolithic chip, so
that you can scale up only the most critical parts.
The so-called Chiplet approach.
2.5D and 3D solutions help enable both directions.
Got it?
Great.
Let's start with 2.5D integration.
## 2.5D History: Hybrid Circuits
The concept of side-by-side die stacks dates back to the 1960s with what are called "Hybrid
Circuits".
Hybrid Circuits combined various devices like integrated circuits and an RF/microwave unit
on top of a substrate.
They were often made for specific certain functions that a monolithic integrated circuit
could not do at the time.
The substrates were often made of ceramic, with bonded wires to carry signals off of
the package.
These hybrids often had lids which can be wielded shut to provide protection from the
outside world.
## 2.5D History: MCMs
From there, we have Multichip Modules or MCMs.
These emerged in the 1980s as direct descendants of the original Hybrid Circuit.
The biggest difference between these modules and their ancestors is their complexity.
They still have a structure where two or more ICs are electrically connected to a common
substrate.
The whole thing is just more advanced.
These interconnect substrates - sometimes also called High Density Interconnects - need
to be fully supportive of the IC, so they tend to be designed and manufactured side
by side.
These modules are classified by how we produce their substrates.
The industry first developed Multichip Modules with the goal of building faster overall systems
- hoping to substantially reduce delays suffered from waiting for a signal to travel from one
die to another - something referred to as "packaging delay".
As computers have gotten faster, such delay went from being a small, insignificant percentage
of clock time to nearly 100% of the cycle.
A few mainframe computer-makers like Unisys tried this solution in the late 1970s for
their high performance computing solutions.
But ultimately, those vendors found it easier to just let Moore's Law to do the work.
Increasing silicon density brought more transistors and functionality onto a single die - effectively
eliminating the packaging delay problem.
Multichip Modules thus fell by the wayside.
## 2.5D Integration and the TSV
Now that Moore's Law has passed us by, 2.5D integration has brought back the ideas of
the Multichip Module but with a modern twist.
In 2009, Dr. Tong Ho-Ming (唐和明) - ASE Group's head of R&D - declared that the industry
might need another intermediate step in order to reach true 3D integration.
Thus coining the phrase 2.5D.
In 2012, TSMC announced its 2.5D Chip-on-wafer-on-Substrate technology, partnering with the big EDA companies
to make it happen.
UMC and GlobalFoundries announced their solutions at the same time - though the actual work
is handled by OSAT partners.
2.5D's key technology again lies within the interconnect.
2.5D uses what is called a "through-silicon interposer", which then sits on top of the
substrate.
These interposers are labeled "through-silicon" because they feature Through-Silicon Vias
or TSVs.
TSVs are vertical interconnects - basically a wrapped conductor passing through a die's
silicon substrate.
The concept is not new.
It was invented by semiconductor pioneer William Shockley all the way back in 1958.
They were called then "deep pits".
TSVs are most commonly made using "deep reactive ion etching" - a method of using corrosive
gases to rapidly etch silicon which had been originally developed for the MEMS industry.
The first 2.5D packaged solutions emerged in the 2010s and include AMD's Radeon Fury
GPUs and Xilinx's Virtex FPGAs.
## 2.5D Advantages
As I mentioned, vendors first chose to implement 2.5D with TSVs because it offers less execution
risk.
However it is more than just a mere stepping stone towards 3D.
A key advantage with this side-by-side arrangement is that we have less space and thermal limitations
on the amount of dies or packages we can stack.
2.5D gives us more physical space to build in more in-package memory capacity, a big
deal for memory-intensive tasks like AI, which face an issue known as the "Memory Wall".
I talked about this in a previous video, but it concerns the efficiency and speed of transferring
data between a chip's logic components and its memory
Lastly, 2.5D does not require a significant re-engineering of the chips in question, so
we can use conventional dies.
With true 3D, the dies will have TSVs drilled through them, so they need to be significantly
thinner.
A significant disadvantage of 2.5D compared to 3D is the additional cost of producing
the silicon interposer itself.
It also represents another point of failure - one of the big issues that AMD and its packaging
partner ASE Group had to overcome was warpage in the silicon interposer.
The industry is exploring new generations of 2.5D integration that get around these
disadvantages by removing the silicon interposer entirely and just having the chips sit right
on top of the substrate.
## Chiplets and 2.5D
Recently, the semiconductor industry has gotten really into chiplets.
By splitting a monolithic die into smaller chiplets, you get better yield and lower costs
without compromising that much performance.
Earlier this year saw the introduction of an open specification for these chiplets - the
Universal Chiplet Interconnect Express.
Such a standard is expected to help chipmakers mix and match dies from different manufacturers.
The standard is backed by practically all the big chip players including AMD, Intel,
Arm, Nvidia, ASE Group, Samsung, and TSMC.
Side note.
We have still yet to see the effects of the US-China chip sanctions on this chiplet interconnect
standard.
There might be some problems with transferring such technology to Chinese firms like Alibaba
- which recently joined.
The 1.0 version of the standard has been defined for 2D and 2.5D chip packaging technologies.
To me, that's an indication that 2.5D has fully matured and we should see a lot more
adoption in the near future.
But we are always looking ahead.
2.5D isn't cool.
But you know what's cool?
0.5 more D. Alright, let's talk about 3D Stacking.
## 3D Integration History
The concept of vertically stacking stuff is not new.
It actually predates the entire transistor era!
We’ve been stacking stuff since the Stone Age!
The first instances of vertically stacking electronic packages that I have been able
to dig up date back over half a century.
In the 1950s, the US Navy had something called the Tinkertoy.
Tinkertoy circuits were large stacked ceramic substrates that could accommodate things like
vacuum tubes.
Obviously, these needed updating.
So in 1958, RCA presented a project called Micro-Modules to the US Army, which wanted
more rugged and mini-sized electronics.
The Army liked what it saw and gave RCA $5 million to work on this.
Micro-Modules were 193 square millimeter ceramic substrates that you can vertically stack.
At one point, the Army had pretty high hopes but the project suffered from high costs.
The coming of the Integrated Circuit as well as the aforementioned 2D Hybrid Circuit offered
far better prospects in the cost department and the project faded out.
Soviets cloned it though anyway.
## Package on Package
In the mid-1980s, vendors demanded new ways to better use the real estate on their printed
circuit boards.
This was particularly important as consumer gadgets got smaller.
In response, vendors created what we call Package on Package, or stacked packages.
Smartphone vendors first adopted stacked packages in the late 2000s.
For instance, take the iPhone 5S, first introduced nearly a decade ago in September 2013.
The iPhone's chip package is stacked.
Its LPDDR3 memory is wire bonded to its package - wire bonding meaning that we use bond wires
to connect the memory to the outside world.
This memory package is stacked on top of the Apple A7 system-on-chip.
The A7 is packaged using flip-chip, a technology in which we flip the chip upside down and
use bumps and balls rather than bond wires to make our connections to the outside world.
This works, but the obvious applies.
Having the package within the package adds some extra heft so you can't stack that many
together.
Furthermore, the interconnect setup isn't as ideal, because signals have to travel from
package to package.
We need to go deeper.
## Die to Die
"Die to Die" 3D integration, or "Die Stacking" involves vertical stacks of unpackaged dies.
Interconnections can be made with either traditional "die to die" wire bonding or TSVs.
Traditional die-to-die wire bonding is quite popular today.
You put one die over another piggyback-style, and then connect them with bond wires.
These have been around since the 1990s and help produce very small, very thin electronics
like those for hearing aids.
However, TSV-connected die stacking is seen as having more potential in the future.
It would let us stack memory dies right on top of a processor die.
This way, we not only can have the shortest possible interconnect route for data to travel
between logic and memory, but also potentially thousands of such routes.
This is the purest ideal, the holy grail.
## Stacked Image Sensors
The first such TSV-connected, non-memory die stacks came out of the CMOS image sensor world.
For their part, the memory industry went three dimensional back in the early 2010s.
I feel like I have to mention that to avoid a flurry of YouTube comments.
I am not sure if you watched my other video about the topic, but today's CMOS image sensors
are made up of what are called "active pixels".
They have a photodiode for collecting light as well as extra transistors for complementary
functions.
One of the big concerns with these active pixels is what we call "fill factor".
It refers to the percentage of physical space in a pixel allocated for collecting light.
3D die stacking lets the photodiode get so much bigger, raising the fill factor to something
closer to 100%.
Sony announced a breakthrough in stacked image sensors in 2005, and then started first producing
them in 2012.
## Overcoming Challenges
Image sensors are one thing.
But there were significant manufacturing issues to overcome before 3D stacked die systems
can hit the market.
First, we needed the aforementioned TSVs.
The industry had to decide how to form these TSVs while avoiding failure modes.
The TSVs are very small - about 2-10 micrometers wide and some 40-200 micrometers long - and
have to go through a multi-step process - etching, lining, filling, revealing, and so on.
Second, we needed ways to cool these dies.
Lower memory temperatures are required to improve the memory's long term reliability.
I noticed AMD put blank silicon chiplets onto their die stacks for heat conduction.
Third, we need to make sure that we can actually rely on these technologies.
Vendors are very concerned about defects generated during these stages.
New systems had to be developed for testing and repairing failures.
It looks like the industry is finally getting closer to bringing true 3D stacked products
to the market.
This includes AMD's Zen 3 processors and Graphcore's Bow AI accelerators, both of which rely on
TSMC's growing advanced packaging capabilities.
And Intel's Ponte Vecchio processor, which uses the company's Foveros technology to stack
chiplets/tiles.
## Conclusion
On December 29th, 2022, TSMC held a rare ceremony in Tainan marking volume production of their
N3 or 3 nanometer process.
The ceremony is a public fulfillment of the company's promise to do this by the second
half of 2022.
N3's development has been a troubled one.
That same month at the 68th Annual IEEE International Electron Devices Meeting, TSMC noted that
its first 3 nanometer process - called N3B - will have minimal SRAM bitcell size shrinking.
About 5%, or 0.95x scaling.
The next version of the N3 family, called N3E, will have no shrinkage at all.
Considering SRAM's shrinkage issues, we will need deeper chiplet-style integration to achieve
improved system-level performance - especially when it comes to meeting memory needs.
TSMC's 3D die stacking offerings are hitting the market at the right time.