"Hey guys! It's Lachlan from Lachlan Likes A Thing. And I'm at my university and we're
having another little headphone meet. So I have a chance to do an ABX test with some
friends. And we're going to be testing the assertion that 128k AAC is transparent compared
to the original lossless file. So we're going to be doing the blind ABX testing in Foobar2000
with 10 trials each. We're going to use the AAC conversion in the latest version of,
version 11 of iTunes. The laptop is in flight mode so all the wireless antennas are turned
off. We are using WASAPI output in Foobar2000. We are using a Headroom Pico DAC and a V100
Vioelectric amp. And I'm just going to ask the guys to choose a headphone of their choice
to use with the amp and a track of their choice. So let's get right to it."
"So our first victim is going to be Shane. And Shane, which headphone are you going to use to do the ABX test?"
"Probably the 650."
"The 650? Can you actually show us the headphone? Thank you."
"Sennheiser HD650."
"And then... and what track have you chosen to do the ABX test with?"
"Chris Botti. 'When Rain Falls'."
"What do you think your chances are going to be?"
"Very slim."
"Very slim. Even with 128k AAC you think your chances are going to be very slim?"
"Yep."
"Okay, alright, so we're going to start 10 trials
starting now."
"So Shane is, Shane is stuck on the first trial here. Shane is
having real trouble differentiating between the two tracks that he knows are actually
are lossless and lossy."
"I would actually not use an open - I think Shane is actually
not able to do this test at the moment because -"
"I'm what?"
"Are you finding it difficult because it's open?"
"I could try a closed one."
"So Shane is finding it very difficult to do this test
with the Sennheiser HD650. So he's going to switch to a closed back headphone
to see if this makes any difference. And he is using the Audio Technica W10VTG."
Offscreen: "It's useless!"
"Useless? Really?"
Offscreen: "You're putting something like, 20 years of technology behind if you use those."
"But they're vintage!"
"So our Shane is taking his time.
Have you, have you done the first trial yet?!"
"Yes!"
"...128k AAC... that's not even half what - iTunes is twice that bitrate."
"Alright, my ears are hurting. It's really loud."
"Alright. Everyone, the moment of truth!"
"Are you ready to see your result?"
"Yeah okay." "For your 10 trials. How do you think you did?"
"Really poorly."
"Really poorly. So remember Shane has to get 9 out of 10 for us to be able to statistically say that -"
Offscreen: "9 out of 10?" "9 out of 10 for us to be 95 - or 98.9% confident"
that he can differentiate between 128k AAC and a FLAC lossless file. So let's unhide
the results."
"And the result is that you got 5 out of 10. Which means the probability
that you were randomly guessing is 62.3%." "That's pretty much on par."
"Awesome. Okay, who's next?" "Who's next?"
"Okay so our next challenger is Thomas. And he is going to be
using which headphone to do the ABX test with?"
"So I'll be trying with my favourite AKG K701."
"Now, which track have you chosen?"
"I've chosen track 3 of the Howl's Moving Castle soundtrack
'Stroll Through the Air'." "And how do you think you're going to do on this
test Thomas? Have you ever done an ABS - ABX test before?" "Uh, yes and no. Nothing very
controlled so... and I've also been using MP3's instead of AAC's. So I don't know how the
sound profile will be different. And whether it's better. It is a newer format. So. I don't
know." "Well, we're going to find out." "Already starting to feel nervous about the fact that
"I'm already starting to feel nervous about the fact that
these are open backed. Because I can hear everything."
"Yeah, that's - I would not do this with an open backed headphone."
"Okay, so Thomas is doing the ABX test again with the FItear MH334.
And because we want to go fully battery powered because he can actually hear the buzz from
the amp from the power supply because of those in-ears we're actually going to switch to
the Objective 2 with the ODAC. And this is completely battery powered so we are completely
isolated from mains power at the moment. So Thomas, go ahead."
"Did I tell you that I was doing ABX tests last night up till 12 o clock
and then I dreamt about ABX tests when I went to sleep?"
Offscreen: "Did you improve?" "It was terrible!"
"I was trying to time this
one little bit and this is the clunkiest interface."
"It is. How many trials have you done?"
"Two so far." "Two."
"Do you think you can do it faster?" "Oh."
"I don't think I can hear a difference."
"Would you like me to just guess the rest?"
"Nil confidence. At all. I feel like these earphones
were poorly suited to it."
"Alright."
"Okay! Alright." "50/50!"
"Moment of truth. Moment of truth. For Thomas with the MH334. How do you think you did Thomas?"
"Abysmal?"
"Abysmal. Okay, alright. And remember Thomas needs to get 9 out of 10 to get a scientifically
valid result. So his results are -
- 8 out of 10!" "What?"
"Which gives him a 94.5% probability
that he did not randomly guess. But did you randomly guess Thomas? In your heart of hearts."
"No. But I didn't have any confidence in it." "But did you hear any differences?"
"The soundstage was a tiny bit, but it was so hard to tell."
"That's interesting. So there you go.
8 out of 10 is just one short of being..." "I'm genuinely surprised..."
"Actually, if it's 5% - yeah, 5.5%, you're .5% away from 95% confidence. So that's pretty good, it does
appear as though you were able to discern some kind of difference."
Offscreen: "Maybe if you tried again you would get 9?"
"No, it's exhausting to do this." "Yeah."
"Okay so Josh is our next challenger. Josh, what are you using for your earphone choice?"
I'll be using the Unique Melody Miracles. Custom IEMs."
"Alright, so, closed. Good idea. We're
actually going to again be using the Objective 2 with the ODAC just to be on battery power.
And what track have you chosen?"
"I have actually chosen 'Platinum Disco' by Iguchi Yuka. Please don't judge me."
"Okay. Alright. Any particular reason why you chose this track?"
"Sorry?"
"Any particular reason why you chose this track?"
"Because I know it well. Basically."
"That's the best way. Okay, so have you ever done any ABX tests before?"
"Not in a controlled environment."
"Okay. And how do you think your chances of success will go."
"Not very well. I'll pretty much be guessing because A) my hearing is terrible and these IEM's
tend to be fairly forgiving of poorly mastered material." "Okay,
"Alright. Well, we'll see."
"Finished trial 1."
"Okay, trial 1. How did you feel? Did you - did you hear any difference
between file A and file B?" "Not really, not much. This is worrying."
"Okay, so were you guessing or did you have some sort of logic behind your choice?"
"I think there is a small, small difference.
I could be second guessing myself, in the drum beats. They sounded a little bit tinnier
on the AAC file. I think. But that remains to be seen whether or not I can tell the difference."
"All right. Let's keep going."
"Next trial, 6." "Are you speeding up because you're starting
to guess, or are you speeding up because you can, you've worked out how to tell the difference?"
"Um, I really think I'm starting to guess at this stage. The more I Iisten, the more
the same the song - each file sounds. I don't know how that works." "So you're getting tired."
"Maybe!"
"Okay, so you have actually completed the trials. Josh has completed his 10 trials."
"Can I look now?" "Yeah you can look now, I haven't...
"So Josh, how do you think you did?" "Terribly!"
"Terribly. Okay, well, are you going to guess a number?"
"Four?" "Okay. So."
Again, Josh needs to get 9 out of 10 for us to be confident that he can tell a difference
between these two files. And the result he got was -
"4 out of 10!"
"At least I was right about that!" "You guessed that correctly!"
"So that gives us a 82% probability that Josh
was guessing randomly. And did you start guessing randomly at some point?"
"Yeah the more, like I said, the more I listened, the more similar the tracks sounded. So. Hmm."
Does this make you feel any different about AAC files?"
"I never really had an opinion about AAC files
to begin with. But you know, I think part of it might be due to equipment. Perhaps,
more analytical equipment, earphones, or IEMs, would allow you to detect the difference.
My mind is still open."
"Awesome. Okay. Alright, well, thank you Josh. And I'm not going to
be able to do any more trials at the moment. But I thought this was a pretty interesting
test. We wanted to test whether 128k AAC was indistinguishable from the original lossless
source, and we kind of had mixed results. Thomas seemed to have a result that was quite
good, it wasn't past the 95% confidence level, but Josh and Shane didn't do so well. And...
You know, I would say from this test that if it's this difficult to discern 128k AAC
from lossless files, 256k AAC at twice the bitrate which is what iTunes offers is probably
a pretty high quality option. Anyway thanks for watching, you can talk to me on Facebook
at facebook.com/lachlanlikesathing or on Twitter @lachlikesathing
Click the Like button if you enjoyed this video and happy listening!