- Today, we're gonna talk through the results
from The Great American Taste Test.
Now, if you missed this,
this was a giant taste test experiment that we did.
We could only do it with people in America,
so that's why it's called The Great American Taste Test.
What we did is we shipped people four different coffees.
We worked with Cometeer, so they were extracted coffees,
which meant that they melted
and diluted the little ice cubes of coffee down,
and it meant that everyone who tasted had theoretical
and identical tasting experience.
And I think this might be the largest preference test done
of this nature ever,
but I can't imagine there's been that many
sort of test tests done like this
where everyone's drinking identical extractions.
We put four different coffees in this pack,
and I explain why we chose these four coffees,
and then we asked three main questions.
I'll give you the results of those questions,
including which one was the most popular.
And then we asked a few more details
that gave us some more insights
that I think are super, super interesting.
The first question we wanted to ask
used coffees A, B, and C,
and ultimately, we were trying to get to the bottom
of how preference around roast level was established.
So, coffee A was a light-roasted coffee.
It was a coffee from Kenya.
Coffee B was a blend, it was a medium roast.
And coffee C was also a blend and a dark roast.
And we thought that was a good question to ask
to establish kind of where preference lies.
Now, we'll talk more about the audience
and the particular demographics that we got to.
Obviously, 4,000 is a big number,
but also a very tiny number,
and so it's the preference of this audience,
and we can't necessarily map it entirely
onto the entire US population.
For context here, the National Coffee Association's
National Coffee Data Trends Report for fall 2023,
used 1,543 respondents.
Though I will say they were, obviously,
much more sort of representative demographically
of the US population than perhaps our data set was.
Then there was a question that was coffee A versus coffee D.
Now, coffee D was a single-estate coffee.
It was an experimental process,
an unusual variety from Colombia.
It was quite a strongly fermented coffee.
People would describe it as kind of funky potentially.
And coffee A was a washed Kenyan coffee.
And we were trying to establish how people felt
about kind of fermented flavors in coffee.
Then at the end we asked, of A, B, C, D,
which was your favorite?
Let's get to those results straight away.
Between coffee, A, B, C, light, medium, and dark roast,
which was the most popular?
This will probably be no surprise
to viewers of this channel, but the most popular coffee
was coffee A in this grouping,
which was the light-roasted coffee.
It had 45% of the votes here.
But coffee B and coffee C each had 27%, 28%.
So really very close between those two.
An almost identical number of respondents, actually.
Quite an even distribution of preference
that I thought was really very interesting.
But then move on to coffee A versus coffee D,
which one do you prefer?
And here, it went to coffee D.
It edged it out.
It had 54% of the votes versus 46.
So not a big sort of difference there.
But, and this was interesting, when you asked people
which was their favorite between all of them,
coffee D won out pretty substantially.
It got 37% of the total vote.
And then, interestingly, again, coffees A, B, and C
received near-identical numbers of votes for them,
each getting 21% to 22% of the vote.
This alone, though,
does not truly tell the story of coffee D.
And we're gonna go back to this
because I think it's one of the most important lessons
from this tasting is around this kind of very fermented,
very unusual coffee that was very popular,
but that does not tell you everything.
The first thing we should talk about
is the kind of gender split
amongst this particular demographic.
72% respondents said male, 24 said female,
3% non-binary, and 1% declined to answer.
And, I think, that definitely skews some of the data.
And when we get into
some of the preferences a little bit more,
you'll see what I mean.
As for age, this one was interesting, as well.
Very heavily dominant in the 25 to 34-year-olds.
Now, we did have respondents above 65 years old
and actually some under 18 years old.
So there was a little bit of a spread around there.
But I would say that, well, over 75% respondents
are between 25 and 44 years old.
And I would say this is an area
where there's really a lot of crossover
with the YouTube channel in terms of demographics.
In terms of people who typically watch the videos
and people who participated in this taste test.
And then one more piece of information was asked
was zip code, so we can show you a kind of distribution
of respondents by state.
I had been curious if there would be
kind of insights based on different states.
We did actually look at whether there were preferences
varying by urban, suburban, and rural zip codes.
And the answer is: Not really.
Not significantly, anyway.
But it was still interesting to see
a kind of distribution of people
from all across the mainland United States.
We did also ask respondents briefly
about some of their coffee-drinking habits.
And some of those, I think, are worth looking at.
Most Obviously, we'd ask them,
"How much coffee do you drink a day?"
And the results, maybe, are kind of surprising.
There's a real peak at one to two cups of coffee a day.
I would've thought
viewers of a YouTube channel like this one,
people participating in something like this,
would be, perhaps, heavy coffee drinkers.
That doesn't seem to be the case,
which is really quite interesting.
I feel like people are probably more aware
of coffee consumption than ever.
I know my own consumption is lower,
much lower than it used to be, but I was surprised to see
two be the most popular answer here
when that question was asked.
We asked people how they brew coffee at home.
And this one was maybe kind of surprising to me, too.
The most popular answer here, by far, was pour over.
63% of respondents said they make a pour-over at home,
which is really quite a lot.
Espresso came in second,
42% of people making espresso at home.
So this, to me, is kind of surprising.
I've got this group of people
who don't drink that much coffee
but have definitely invested in their home setup.
I always presume people who drink espresso at home
maybe drank more coffee.
Maybe that was a stupid thing for me to presume.
This is definitely insightful.
I was pleased to see the French press in there at 20%:
A popular brewer, and rightfully so,
and then a pretty even distribution of other things.
9% using some sort of pod system.
4% instant coffee.
Some, you know, using bean-to-cup machines, though not many.
If you consider how many bean-to-cups are out there,
that's maybe an audience, in future,
it would be interesting to kind of access
that we didn't really access here.
Cold brew also surprisingly popular, but only to me,
because I live in a cold wet place
and never really think about cold brew that way.
Now, one interesting question that I really like
is we asked people to rank from zero to 10
their level of coffee expertise.
We didn't really give guidance here,
and so there's a broad range of results.
There's a real peak at sort of six to eight.
People who answered six and seven,
five and eight, also popular.
There's a kind of clustering there.
There are people that answered 10
and there are people that answered one.
And that data is really useful when we look at it later
'cause it shows something really cool.
So let's talk about coffee D,
because this one, definitely interesting.
And not hugely surprising to me
that this unusual, funky, fruity coffee came out on top.
It was, you know, I think, clear and unusual and different
and kind of fun,
and I think a lot of people gravitated towards that.
And so you'd say, "Okay, people really like naturals,"
but that does not tell the whole story.
If you look at the preference data for coffees A, B, and C,
you'll see they're all actually quite similar:
They have a peak in the middle,
and then they decline at either side.
Some people really like them, some people didn't.
Most people were kind of like, "Yeah, okay."
Coffee D is sort of the opposite of that.
Yes, it has a lot of, "I really like this,"
but it also has the most, "I really dislike this."
So the key lesson here is that D was popular
but highly, highly polarizing.
And, actually, more interesting still,
when you break that coffee down by gender response,
it gets really interesting.
Now, we only have enough data to split it out
into people who said male or female,
but this was a coffee strongly preferred by men
and actually strongly disliked by women,
and that is very surprising.
I had assumed that preference was kind of agnostic
in this particular regards, but it seems in this test, not.
And, you know, if you look at the data,
there's some really different distributions of preference.
I think that's really notable.
There is a clear implication for this,
which is if you serve these kind of coffees,
that's great, they're really useful.
And if you serve naturals and promote naturals,
that's really useful.
People really do like these kind of flavors.
But this should not be on a kind of default option.
If you serve batch brew,
do not put natural processed coffees
that taste kind of funky and fermented on your batch brew
because there is a good chance that a decent portion
of your audience will not enjoy that coffee.
They just don't like those kind of flavors.
Now, this is, I think, a reasonably good reflection
of kind of the American population.
I've seen this when we've done previous tastings
in other populations, too.
It's not uniform.
There are certain populations
in different parts of the world
that have a much higher appreciation
and a broader acceptance of very fun, funky,
and kind of fermented flavors.
But I would say, if you're in the US,
I would say probably the same as in the UK
from the data that I've seen in previous tastings,
it's not a default option.
There - if you wanna do a natural on batch brew,
you just need an option that isn't that.
You just can't presume that everyone will like this.
I think, you know, looking at kind of the washed coffee
as a kind of counterpart to this, no one really disliked it.
Even as much as maybe less people really liked it,
no one was disappointed or upset by it.
And I think that's the lesson here.
Which is, those kind of coffees are very, very useful.
I think there's definitely a market for them.
But, but it's not everyone.
Diving a little deeper into this,
I almost wanted to say something
about the sort of age differences here.
Because, if we look at preference by sort of age,
it's really interesting,
and there's two potential explanations of this.
As you look at the data here, you'll see that coffee D
in younger demographics is much, much more popular.
And as you cross kind of 45 into 50 into 60 years old,
it gets less and less and less popular,
and darker roasts get more and more and more popular.
Thinking about this, I can offer two potential explanations,
and I'll tell you which one is my favorite.
Firstly, there's the idea
that your taste buds change as you get older
and you find yourself drifting towards darker
and darker roasted coffees.
I'm not sure that's the case,
'cause I have a competing hypothesis here.
I would say that the benchmarks
for kind of typical coffee flavor
are set in your relative youth.
When you start drinking coffee,
those kind of formative years of coffee drinking
set your expectations of what coffee can
and should taste like.
I think if you started drinking coffee 30 or 40 years ago,
you were likely drinking darker roasts.
You were not drinking light, washed coffees.
You were not drinking weird fermentations.
That's not a familiar coffee flavor to you.
And I think what you start drinking
is probably what you keep drinking for a lot of your life
because that's the thing that you learned to like.
I think if you start drinking coffee today,
your idea, your expectation of coffee's flavor
is gonna be so much wider.
You're gonna be much more accepting
of extremely fermented and unusual coffees
because they've kind of always been there as an option
since you started drinking coffee.
That, I think, is my sort of theoretical
proposed explanation of this particular data.
But it definitely fit with, you know,
my experience of the world,
which is older people like darker roasted coffee.
But I think the "why?" is still to be explored,
and definitely super interesting.
Remember when we talked about people's
sort of coffee expertise and ranking themselves?
Well, if you break down their preferences
by their expertise level,
you see something, I think, fascinating.
First thing that I wanna remark upon
is that the people who ranked themselves one
had the kind of most equally split preferences
of the coffees.
And it sort of implies
that certain preferences develop over time.
And certainly, if you look at this data,
you would say the deeper into coffee you get,
the more you enjoy both fermented coffees
and the more you enjoy lighter roasts, to a point.
Because it looks like once you get to sort of
a self-ranking of eight out of 10,
that is peak appreciation of fermented flavors.
And then what we see,
in people who rank themselves nine and 10,
is a decrease in preference for fermented flavors
and a small increase in preference
for sort of washed coffees.
This, I worry, totally fits into my sort of lived experience
of the coffee industry as a whole.
I've worked in coffee for maybe 20 years now.
I know a lot of people who fell in love
with fermented coffees,
with natural coffees quite early in their career,
loved, loved, loved them, and then for a variety of reasons
get to the point where they just don't
really enjoy them as much anymore.
I know lots of coffee professionals
who are kind of industry veterans
who just don't wanna drink naturally processed coffees.
I am totally potentially making the data fit
my lived experience here,
but it is interesting kind of data to look at.
Obviously, by the time we're ranking,
you know, an expertise of 10 out of 10,
that's a much smaller subset here.
So, the numbers involved are relatively small,
which does skew your data.
But it just was interesting to see
that, you know, your appreciation
climbs and climbs and climbs, until a point,
and then it kind of stops,
and it does not continue to grow the same way.
That, I thought, was worth looking at, mentioning,
if only to talk about how people in the coffee industry
get kind of weird in a particular way over time.
Couple of quick ones that I think are just interesting:
If you look at the breakdown of coffee favorite
by how you brew coffee for yourself,
you'll see that the pour-over drinkers
strongly prefer lighter roasted coffees,
and unsurprisingly, latte drinkers prefer
the sort of medium and dark roasted coffees in comparison.
I mean, pour-over really has a very strong preference
towards coffee D, those kind of fermented flavors,
but also a pretty big preference towards coffee A,
that washed Kenyan coffee.
It doesn't kind of carry through the whole way.
You don't see in espresso drinkers
or kind of shorter milk drinkers
a preference for medium or dark-roasted coffees.
Actually, in this particular data set,
a little bit more towards the light end.
But, you know, I don't know if, you know,
the idea that your drink of choice
is a good reflection of who you are
is really, you know, shown in this data.
I think that was something we talked about
or was joked about for a long time,
like you are your coffee order.
I'm not sure we're seeing that strongly in this data.
Now, I wanna talk about stated preference
versus revealed preference.
We asked people what kind of, you know,
taste descriptor words they like in their coffee.
Did those descriptors match up with the particular coffees
that exhibited those flavors?
And I would say the answer is yes.
People who selected 'fruity' as a choice
very strongly preferred coffee D and then coffee A.
People who liked 'chocolatey'
mostly preferred the medium roast and the dark roast.
And so, you know, I think those kind of words
turn out to correlate really surprisingly well
to the kind of taste experience.
We've talked negatively, often, as an industry,
about taste descriptors being kind of useless
or, you know, pretentious or confusing or frustrating,
but I think when you use the big broad category ones,
they mean something to people
and people do know that they like fruity,
they know that they like chocolatey,
they know that they like nutty or bright.
Those kind of words really strongly correlate with coffees
that I would say exhibit those taste characteristics.
But
when we come to roast level as a descriptor,
the correlation, hm, goes a little bit wonky here
in an interesting way.
And, yes, people who said they like light roasted coffees,
in this, did show a preference for coffees A and D,
the light roasted coffees.
However, people who said they like medium roasted coffees,
well, their most popular coffee was actually
the dark roasted coffee.
And people who say they like dark roasted coffee,
well, their most popular coffee
was the medium roasted coffee.
And I think it's not that surprising
because medium and dark are very fuzzy words these days
in the coffee landscape.
You know, 'specialty medium' to some people
is still quite light.
'Specialty dark' might be, to some people, medium.
It's not a very helpful word to use,
and it actually doesn't correlate as neatly
and as strongly as you might hope.
In an ideal world,
I think a little bit of a darker dark-roasted coffee
in the tasting sort of kit might have worked better
to split that out.
But I can't help but feel that 'medium' and 'dark'
are not that helpful as words, as guidance words,
as selecting words, as preference words
in the coffee world right now.
Now it's time to wrap this up,
and I should, I think, criticize this project to some extent
because I wanna think about
how it could be done better in the future.
Firstly, I think, 5,000 kits
was the kind of limit logistically
for this particular project in the timeframe that we had.
I wish we had been able to do more,
and I know a lot of people were frustrated
that they couldn't take part, and I apologize.
It's hard to really know the sort of level of demand
for these kind of things.
But 5,000 kits yielding 4,000 answers,
it's an enormous amount of data,
but I'd love 8,000, 12,000 responses
just to kind of broaden it a little bit more
and kind of broaden it outside of the reach of the channel,
specifically into a kind of wider population.
Ultimately, this first and foremost,
was just supposed to be a really fun tasting experience.
I think a lot of us had fun doing it.
I thank all of you that took part,
that tasted along with me.
That shared the data that you did
that allowed us to kind of have this insight.
I'm extremely grateful to all of you.
I'm very grateful we got to do some citizen science,
and I get to share these results with you.
I should be clear, this data is open access.
There will be a CSV file of every response,
anonymized, obviously,
available to download in the description down below.
Anyone can look at it.
If you find some really interesting data in there,
please leave a comment.
But yes, I'm grateful that data is available.
It's open access.
It should be fun to just kind of have a look through.
I'll just warn you, it's a lot.
It's a lot.
It's, like, 4,000 rows deep.
There's a lot of columns going on, too.
It's a big spreadsheet.
Thank you also, again, to Cometeer
for doing all of the hard work and legwork on this.
The logistics on this is no mean feat
for this many sort of kits.
And also, crunching the data afterwards.
For people who tried the coffees and want more,
there is a discount code in the description down below.
That gives you, I think, 20 bucks off your order
if you wanna try it.
So, thank you to Cometeer.
And, lastly, thank you.
If you've thoughts on this,
if there's things you think we should look at in the data,
if there's any insights that you wanna share,
if you took part and wanna share your experience with it,
I would love to hear from you down in the comments below.
But for now, I'll say thank you so much for watching,
and hope you have a great day.