- There we go.
10 gig networking on an M1 Mac mini that's just, wow!
Ah, well, maybe,
the thing is just because a port is present on your computer
doesn't mean that it will operate at full speed,
and it certainly doesn't mean
it will operate at full speed all the time.
USB for example, is rated by the speed that is shared
across multiple ports,
high speed I/O and SSD slots often share resources,
meaning that both of them will slow down
if you use them simultaneously.
And in the early days of gigabit networking
basically no motherboards
could run those ports Apple's speed
because they were connected to the lethargic PCI bus,
you know the old non Express PCI bus.
Of course, we only know these things
because vendors like Intel published block diagrams
of their platforms to help us understand their capabilities.
Apple on the other hand does no such thing,
meaning that if there are internal bottlenecks,
the only way to find out about them
is to have your performance drop
while you're in the middle of doing something.
That makes today's adventure interesting for two reasons,
one, it'll tell us
if Apple delayed this version of the M1 mini
perhaps over performance concerns
and two, it might give us some fresh new insight
into the black box that is Apple's M1 SoC
and it also might give us some insight
into our sponsor, Honey.
Honey is the free to use shopping tool
that helps search for some of the best promo codes
on lots of your favorite sites.
Get it today a joinhoney.com/LTT.
(upbeat music)
The M1 Mac mini blew us away not just with its performance,
but also with its price.
It beat out every previous Intel Mac
and single-threaded benchmarks and a Ryzen 5 5600X
one of the fastest single thread desktop CPU's in the market
all without breaking the bank.
It didn't have the cores to back that up
in heavily multithreaded tasks,
but so far that hasn't generated many,
if any user complaints
thanks to M1's tight integration with macOS
and of course the availability of the Mac Pro
if you really need more horses.
What has generated complaints is the I/O,
with just two USBs, two Thunderbolt ports,
and a single one gigabit ethernet jack
M1 mobile pedigree seemed to be holding it back
for people like me who loved the CPU performance
but need to work with heavy video footage
or other shared resources over a network.
If one gig wasn't enough
you had to use a bulky expensive dongle
and use up one of your Thunderbolt ports,
or you had to buy into a dead end Intel platform,
which was so disappointing for me personally,
after seeing Apple push 10 gig forward
over the last few years,
first with the iMac Pro, then with the Mac mini,
but it turns out I just didn't have enough faith,
in my hand right now is a shiny new 10 gig version
of the M1 Mac mini
and we're gonna be running Iperf on it as a server
to give us full control over what other ports
or devices might be hogging PCI Express bandwidth
causing potential bottlenecks.
So to start,
we were running an absolute best case scenario here
with nothing plugged into it
other than networking and power, not even our display.
Once I come over here and have a look, that is...
Dang! That is pretty stable.
9.25, 9.45 gigabits per second.
Not too shabby 'cause remember guys
10 gigabit is the line speed,
and it's perfectly normal for the actual data speed
to be a bit lower.
What's not normal is the price
for our 40 ounce water bottles on lttstore.com,
they're the same as 21 ounce, what a deal.
Now let's up the ante and connect a Pro Display XDR
but Linus you might ask,
why would plugging in a monitor
affect how much bandwidth is leftover
for something like high-speed networking?
I'm so glad you asked, the M1 Mac mini is notable
for being the first mini with DSC
or Display Stream Compression.
And what that does,
is it allows it to have enough leftover bandwidth
to run the integrated USB 3 hub on this monitor
at full speed.
Previous Intel-based Mac minis ended up running
either not at the full 6K resolution
or in an uncompressed tiled mode
that some users have reported actually took this hub
and knocked it all the way back to USB 2 speeds.
That's because uncompressed, this monitor
will suck up about 34 gigabits per second of bandwidth.
To be clear, I'm not actually expecting this
to throttle our network speed.
It's just an excellent illustration
of how a system has a finite amount of resources
that needs to be carved up.
Also, now that we've got our display up,
you can see that the terminal on our mini
matches our SSH session
so no smoke and mirrors here ladies and gentlemen.
Well, let's go ahead and run it again, shall we,
and as expected we are exactly the same.
I mean, no computer manufacturer in their right mind
would rate the speed of their ports
assuming that no display has plugged in
especially not a display they make.
But yeah, we're gonna kick things up a notch
and to do so Anthony repurposed,
some of these shell script that he wrote previously
that constantly writes data to an SSD
and then spits out speed info regular intervals
for graphing purposes, and he targeted it
at this high performance Thunderbolt SSD.
Maybe with this thing running,
we'll finally start to see some bandwidth sharing
in our Iperf test.
All right, Apple, all right, I'm impressed.
Our SSD right now is pulling over 10 gigabits per second
in reads putting us darn close to the,
up to 40 gigabits per second number
that's advertised for these ports
but all that tells us so far
is that the two Thunderbolt ports don't share bandwidth,
so let's continue.
Remember the USB hub I mentioned
on the back of the Pro Display XDR
well we're gonna load it up with USB-C SSDs.
We're gonna run that SSD script
on all three of these at the same time,
then we're gonna run our network test
and see if we can make this poor Mac mini cry.
And good news, sort of, our SSDs did slow down,
that means we managed to find a bottleneck
but it's in the USB hub of the Pro Display XDR
which presents us with a couple of problems.
Number one, that doesn't actually tell us anything
about internal bottlenecks of the M1 Mac mini,
and number two is we are fresh out of USB-C ports
to plug SSDs into.
Unless a Thunderbolt dock, ah, yes!
We need a Thunderbolt dock and more SSDs. (laughs)
We're gonna plug all these in
and then we're gonna run the script on them
and then see how you like that.
There we go.
Five SSDs plugged into the hub and then for good measure
an additional one another Thunderbolt 3 one
plugged into the Thunderbolt Daisy-chain.
Guys, if that 10 gig ethernet
is stealing bandwidth from somewhere
we are going to find it.
Our pre-planning did not have this actually go down at all.
We figured it out,
one of the discs was not showing up properly
and running the command to a missing target
was causing something,
I don't know, maybe some kind of CPU overhead.
Anyway, now that all the disks are showing up
we are getting the results we expected,
which is anywhere from 9.3 to 9.42 gigabit per second
and this is even with all of the drives running.
- The CPU usage while we're just doing this is around 18%
and that's with it all the drives plugged in.
If I add that drive that's not plugged in,
what's our CPU usage now 32,
36 significantly higher.
- Interesting.
- I didn't test this on the bench, I didn't expect this.
- Hey, it's done.
- 40,
77. (laughing)
- Okay, Wow!
Way to write a script that hurts computers
when you do it wrong.
But while our side adventure interesting,
you know, Hey it turns out runaway CPU usage
is going to affect network transfer performance.
It wasn't really the point of the video today.
So let's refocus and talk about what we've learned so far.
There's a few things actually.
So one, we now know that the 10 gig M1
is legit non-shared 10 gig,
no switch chips, no controller hubs
just a whole Gen 4 PCI Express lane.
That's why nothing that we did was able to bottleneck it.
So clearly Apple's delay had nothing to do with performance
and you can buy this puppy with confidence.
That's cool, but we're not done,
diving deeper while our theoretical max speed
when testing the drives individually
works out to just over 58 gigabits per second.
It capped out somewhere around 50 in the real world
and that means that we did throttle somewhere.
The SSDs connected to the Pro Display XDR
were obviously going to throttle as they were being crowded
by that chunky 13.7 gigabit per second
display port a signal.
But the Thunderbolt 3 hubs throttling is another story,
and this one comes primarily down to the fact
that each Thunderbolt 3 port appears to be capping out
at around 20 to 22 gigabits per second.
But hang on a second Linus,
Apple said each of these ports
could do up to 40 gigabit per second,
and you said they didn't share bandwidth, what gives?
So remember how our 10 gig NIC is expected to be less
than 10 gigabit per second in the real world?
Well, the same is actually true for Thunderbolt 3,
USB for whatever the hell this thing is.
So around 7.6 gigabit per second of that
is set aside for our display port,
which are Thunderbolt 3 dock has,
and then the rest of it gets eaten up by the signal encoding
bring us down to a theoretical maximum
of only around 26 gigabits per second.
And that's close enough to our numbers
that the difference probably comes down
to the fact that we're going through busy hubs
rather than attaching directly,
hence the wavy patterns on the graphs
compared to the individual drive results.
That means that if you've got applications
that are sensitive to I/O variants, like audio
or anything involving real-time data
you would do best to directly attach
or use only one high-speed device
on a hub or dock at a time.
Of course, for most people this is academic
and it won't make a difference real world.
Well sort of because M1 is still I/O limited
and it has affected the port options on Apples M1 products,
like for example, the new iMac which we'll be reviewing,
so make sure you're subscribed.
The question just becomes, how limited is it?
What it looks like is we've got two PCI Express Gen 4 lanes
going to Apple's Thunderbolt controller
which steps down to two Thunderbolt three channels,
then we've got one Gen 4 lane for the 10 gig NIC,
there's 10 gigabits to go around for the Type-A ports,
so that's another lane
that is maybe shared with the Wi-Fi 6 adapter,
and then there's an HDMI port connected directly to the GPU
via an internal display port to HDMI converter.
That's your I/O, four lanes accounted for.
We ran the I/OLGA terminal command
to try and find out for sure
but while it does show device hierarchies
it doesn't show how they're connected to the main system
just that they are connected.
Of course, most PC systems have a lot more
than four PCI Express lanes.
Surely M1 does as well then right?
Well, not necessarily.
You might think it would have another four lanes at least
for its SSD, but it doesn't.
Remember how the T2 chip that we loved so much
was the storage controller
for the final generation of Intel Macs
and then it just disappeared with the M1.
Well, the reverse engineering effort
for getting M1 running on Linux
has actually discovered that Apple wasn't kidding.
The M1 SoC has its security chip built right into it
and it's controlling the SSD directly
through some kind of memory map arrangement,
not through PCI Express
and you got to remember NVMe is a protocol,
so while we associate it with PCI Express
it doesn't technically have to run over PCI Express.
All of which is to say then that the M1 Mac mini
is probably near the limits of what Apple can manage
with the M1 SoC but it's got far more I/O
than we initially gave it credit for
because remember we haven't even accounted yet
for whatever bus they're using for the internal GPU.
So the reason they held off on the 10 gig version
was probably more to do with logistics or strategy,
but whatever it was Apple has done a lot here
with little and that's pretty admirable
kinda like how Apple used to be
back in the good old days and kinda like my segues
to sponsors like Micro Center, get the best prices
and best selection on computer hardware
and everything else technology at any one of Micro Center's
25 locations across the United States.
Micro Center's custom PC builder
will help you spec out the best PC for your budget
by making sure all your parts are compatible
finding stock that's available
at your nearest Micro Center location
making it so you can just add it to your cart
and arrange same day in store pickup.
For a fee you can check the box marked Same Day Pro Assembly
to have a Micro Center expert technician
assemble the PC for you and if you want help
deciding what parts to put in your new custom gaming PC
just join the new online Micro Center community.
It's a great place to talk tech with other enthusiasts.
Check out the link in the description
for a free pair of wireless Bluetooth headphones
valid in store only no purchase necessary.
So thanks for watching guys,
check out our review of the M1 Mac mini
when it first launched
to get just a little bit more of an appreciation
for how much of a game changer
this modest piece of Silicon could be.