(rock music)
- Over the last couple of days
I've been doing what I do around the studio here.
That's getting deals,
doing videos,
cutting new builds,
whether it be potatoes or brand new computers.
And then this new story pops up.
It was set loose by a Steve over at Hardware Unboxed,
and then after that followed up by the tech Jesus himself,
Steve from Gamers Nexus.
They made some really good points.
Hardware Unboxed even took it a step further
and put the benchmarks that are posted on
the Principled Technologies commissioned benchmarks
against their own benchmarks ,
and found these numbers to be seriously misleading.
This is why I'm making the video here today,
because these benchmarks only aren't just misleading,
they differ from what we would expect out of companies
from the pre-release benchmarks.
The numbers that they're used to posting up.
When Nvidia or AMD and Intel usually have
these prerelease benchmarks,
they kind of show their CPUs in the best possible light.
In other words,
they cherry pick some benchmarks,
where their products will be running better
than the competitors.
But the biggest problem this time around,
is that the competitor's product,
and that's AMD and mainly the 2700X because
this is going to be the main competitor to the 9900K,
this CPU has been seriously hemorrhaged
and debilitated in these benchmarks.
It is disgusting when you look at it really,
and apparently Intel commissioned these benchmarks
to a company called Principled Technologies.
In this case,
I think Intel ought to decommission
these benchmarks straight away.
That's because they're misleading,
and the average person that goes up
and looks up some 9900K benchmarks on Google
might go then and see these benchmarks and be like,
"Oh, I better get one of these
because I'm going to get 50% extra performance in games,"
when we know that as reviewers this is completely false.
Most people with a 1080 Ti and even
the RTX 2080 and 2080 Ti's are going
to be playing higher resolutions,
whether it be 144 hertz,
1440p or 4K,
and you're going to see sometimes as little as
a two or three percent difference.
They cut the eight core,
16 threaded $300 CPU in half.
I feel like the 2700X is currently aimed
as not only as the best value proposition,
but it's their best gaming CPU too.
I'm looking through these benchmarks,
there's a total of 19 of them conducted.
They were then posted by a website called PCGamesN,
where they then put them in graph form.
You can see in some of these benchmarks
there's a big difference between the 2700X
and the 9900K.
Also if you look closely,
you'll see that the 9900K is also sometimes
beating out the 8700K and the 8086K,
which shouldn't be happening.
So we go back to this original paper,
we start doing some more in depth reading,
and we see that the settings are just all over the place.
For starters,
on the 9900K they're using the Z390
and they haven't really disabled anything.
Then you move over to the 8700K.
They disabled multi-core enhancement,
so they're making sure these run at lower clocks
than the 9900k.
This is within Intel's own CPU line-up.
Then we move over down to the 2700X
and again everything's been disabled.
Then we move over to the memory timings,
which is so important
for AMD's CPUs,
especially when it comes to benchmarking video games.
AMD's rise in CPUs love a combination
of higher memory speeds as well as lower latencies.
So I was a bit surprised to see here that
they just locked in a lazy,
manual,
2933 megahertz without changing the timings at all.
Then they went to Intel's side,
and they enabled the XMP profiles,
essentially dropping the latency down on the memory,
and then capping that to 2666 megahertz as if to say,
"Hey, we gave AMD an advantage here guys,"
when in fact,
they actually gave Intel another advantage
on memory speeds too because Intel,
if you guys didn't know,
it actually likes lower latencies for gaming more so
than higher speeds on the memory.
AMD on the other hand,
especially with Ryzen,
it likes as we just said before a combination
of higher speeds and lower latencies.
You couple them together,
you're going to get better FPS to a certain point.
Moving through these benchmarks,
at least the ones that show the biggest discrepancies,
at least from what I know in testing some of these games
and what we're seeing here,
Ashes of Singularity,
this game uses up to pretty much a maximum of 16 threads.
Isn't it ironic that the 9900K has 16 threads?
Past that,
you'll see the core utilization really drop off in this game
and it's not that good of an optimized game.
It's actually got inherent input lag in the game itself,
and I've tested this in the past,
and hence why I've dropped it
from my benchmark suite of games.
But here's where we saw the biggest difference between
the Ryzen 5 1400 and the 9900K.
Moving through some more of these benchmarks,
we can see big discrepancies in Fortnite,
PUBG,
FarCry 5,
Civ 6,
Rainbow Six Siege,
and Rise of the Tomb Raider.
The biggest tell tale sign is that these benchmarks
should not be taken seriously,
is that when we look at the 2700X numbers
also compared to the 2950X and the 2990WX,
I've tested all these three CPUs in the past,
and the 2700X does the best for gaming out of AMD's CPUs.
I was really surprised to see it losing
to even its big brother counterparts
which have more cores and threads
and are more suited to streaming and also productivity.
We also take things back on these benchmarks
and we look at for instance Rainbow Six Siege
versus PUBG with their own numbers with the 8086K
versus the 8700K,
and we can see that there's just inconsistencies
within their own benchmarks themselves,
which just makes me really scratch my head
as a reviewer who actually does proper benchmarking
and tests things apples to apples,
because I don't believe these CPUs
were even tested apples to apples.
But wait,
the ridiculousness,
(laughs) the ridiculousness,
- [Man Offscreen] (mumbles something)
- Is it?
But wait,
the ridiculousness doesn't stop there.
If we look at the way they've tested these games,
they've used the in game benchmarks whenever they could,
but then they've also used FRAPS
and just reported average FPS numbers.
There's no .1 and one percent lows,
which are arguably way more important
in a CPU test for gaming than they are for a graphics card.
I'm actually fine with people
using average and minimum FPS for GPU benchmarks.
But when it comes to CPU benchmarks at high frame rates,
we want to know if the CPU's stuttering
so all the variables with that CPU
and combination of testing
can be identified as maybe flawed or whatnot.
Alls we're seeing is average numbers
across FRAPS and in game benchmarks,
and as Steve from Gamers Nexus pointed out too,
they're only using a median benchmark.
In other words,
they're only just picking one benchmark
out of the who knows how many runs they did.
They could have just run these benchmarks
once for all we know,
they're just that inconsistent.
Here's where things get more frustrating
for me as a reviewer.
I'm under NDA so I can't refute these claims
until the 19th of October this month,
but I want to send an open message to Intel,
and that is please decommission these benchmarks
because they're just not accurate.
They're not even cherry-picked benchmarks,
which is kind of as I said in the intro
expected of major companies like yourself,
Nvidia and AMD.
Everyone knows you do it,
but when you're hemorrhaging the competitor's products
it's just not fair and it's misleading to consumers.
The average guy could search up numbers for the 9900K,
come across these benchmarks and these fancy graphs,
and think,
"Wow, I'm performing 50% better in games,"
but then,
when you go to the bottom of the page,
there's a frickin' disclaimer to say
"Hey, we don't take any accountability
for these inconsistent benchmarks."
So at the end of the day,
as someone who takes pride in my benchmarking
and the way I present numbers to you guys,
I just find this so misleading.
We're all guilty of making mistakes,
when I make a mistake I'll tell you guys.
If it's a big enough mistake
that I feel it could mislead people,
I'll take my video down.
Anyway guys that's about it for today,
I hope you enjoyed this video.
If you did,
be sure to hit that like button,
but also let us know in the comments section below
what you think about this whole
9900K commissioned benchmarks.
I just think it's
(sighs)
it's not right,
I mean Intel don't have to do this.
I'm sure the 9900K is going to be a solid product,
they're bringing back soldering too
between the IHS and the die,
so it feels like they're listening
to the consumers and reviewers like myself
who have critiqued them in the past for these things.
But then they go and commission these things
and put out a statement saying that this data is consistent
with what they're getting in their own labs.
I really don't know,
uh,
what was going on with that reply either.
I mean seriously,
just decommission these benchmarks.
We'll get you guys out some solid numbers day one
that you can actually rely on
and make a solid purchasing decision.
When it comes to the 9900K,
eh.
But anyway,
when it comes to opinions,
I'm sure everyone has one.
I implore you guys to reserve yours
until the product is released
and we can see real numbers from trusted sources.
Anyway,
I'll catch you in another tech video very soon.
Peace out for now.
Bye.
- [Voiceover] Trusted sources
like this new merch right here.
Ooooooh yeahhh.
You gotta get some Tech YES City retro shirts baby.
Link in the description below.
(electro pop music).