(audience applause)
- Hi everyone.
So I know this is probably a huge cliche since probably
every single speaker says this, but it's a huge honor
to be here.
I was sitting in your seats about 10 years ago which seems
like not that long and yet quite a long time.
My class, the class of '06 just celebrated our 10 year
reunion here like a couple months which was a lot of fun.
So it's really really great to be here.
I remember when I was sitting in your seats and you know
I had just come from Texas, that's where I was raised
and I had done some web design in the past,
but I didn't really know anything about Silicon Valley,
didn't know anything about this kind of tech hub
and what it means to have a career here and startups
and all of that prior to coming to Stanford.
And I remember sitting in the audience and listening
to the stories of people who had come before me
and in particular Marissa Moyer and Jeff Fitzpatrick.
They'd stand up here and they would take about their
adventures after leaving Stanford and all of the great
things they got to go and build and work on.
And I was just like wow, this is super super exciting.
And you know, now I'm excited to be here talking to all
of you guys and hopefully sharing a couple of my stories
along the way.
So as you guys heard, I was here, I did my computer science
degree, my bachelors and my masters and I was also lucky
enough to be part of the Mayfield Fellows Program.
So how many of you guys know about
the Mayfield Fellows Program?
Okay so for those of you who don't know, this is an awesome
program that is really about diving into entrepreneurship
and so one part of the program is that between your junior
and your senior year, you go and get an internship
with a startup somewhere in the Valley.
And along the way as we're interning at our respective
companies we're learning about entrepreneurship, we're
taking classes we're doing case studies.
And it's fabulous, it's awesome, it's taught by Tina.
I had a lot of fun doing it.
So at the time the start up that I chose was actually
Facebook and you know this is a little bit of a little
cheating because in 2006 Facebook was already
about 100 people, so it was a startup but on the larger end
of the startup.
But it was something I was really excited to work on.
It was a product that I and all of my classmates at the time
used very religiously.
So I went there and I remember my first day at Facebook
in fact, I was signed up to be an engineer and work on
some of their photos features and I had a mentor
and her name was Rudgy and on the first day she said,
okay there's been a change of plans I'm switching jobs,
I'm not going to be an engineer anymore, I'm going
to be a PM, I don't really know what that means for you,
but let me introduce you to this pod of people over here.
This is our design team, why don't you sit with them
and you know, chat with them.
I've got to go run.
And so she left me in this pod with this group of designers
and before that, I'm not a designer by trade, I didn't
really know anything about design, but I got to talking
to these people and I sat at that pod.
And low and behold, fast forward 10 years
and now my career's in design and when I look back on it
it sort of seems really quite random how it happened.
But for me what was so exciting about design was the chance
to work at the forefront and thinking about what are
the people who are going through the experience
that we're building, what are they feeling,
what are they thinking, how are they able to understand
the products that we've built.
And so to me that's what was super super fascinating
about design and I also wasn't that crazy because
at the time the designers that we hired were also
our front end engineering team, so it was a little bit
of both designing and a little bit of coding.
This is back in the world where we were still operating
with just websites, so you just had to do some CSS
in JavaScript and front end PHP and it wasn't nearly
as complicated of an ecosystem as it is today.
But we got to building and one of the first things I worked
on was photos and photo products, I also remember one
of my first big launches a couple months later,
which was Newsfeed back in September of 2006.
Over the years I've worked on things like the Facebook
platform and ushering a bunch of applications built
on top of Facebook including a lot of games,
back in the era of 2008 and 2009.
I worked on Profile and Timeline and I worked on Newsfeed.
And today the team that I lead, we work on design
for all of the core features of the Facebook application.
So when you go on Facebook and you want to catch up
with your friends, you want to share, you want to watch
videos or join a group, those are a lot of the things
that my team works on.
So all that said, that's a little bit of an intro
into kind of how I got here.
The thing that I think has been the most fascinating for me
as I look back on all the things I've worked on
is the moment when you're starting a project and you're
thinking about here's this awesome new idea that I have
and I can imagine how it's going to work and how people
are going to use it and I want to build it right now
and you rally a bunch of other people together with you
and you guys are all excited about this idea.
That's how everything happens.
That's how any idea ever sees the light of day.
But the question that I found super fascinating, is how
do we know at that point in time whether this is an idea
that's going to be successful?
How do we know after it's all said and done and we've
worked our nights and weekends and we've gotten it out
to the world, will it actually be something that people
find valuable and that they'll find easy to use
and that they'll find well-crafted?
And being at Facebook, we've had our share of features,
some of them that have gone on to be great success,
and some of them that haven't.
And a lot of times at the beginning, you know, it feels
like there's really no way to tell.
It feels like you're kind of rolling the dice a little bit.
And this is the thing that I wanted to really study
and reflect on over all of the different products
that we've built.
Is what were the patterns for the things
that were successful, are there ways that we can tell
as we are building whether this is something that's going
to work or not.
And what came out of that, and a lot of discussions
and a lot of postmortems and a lot of just looking at
the things that we had built, was a framework of just
three simple questions at Facebook that we now use
to ask ourselves whether the things that we're building
feel like that they're on track.
And they're three very simple questions because you know
it wasn't going to be a manual that everybody was going
to memorize and understand, but it was three questions
that we wanted everyone at Facebook, no matter what their
role was or what they worked on, whether they were engineers
or designers or product managers, to keep in mind
when they're having a team meeting or whether we're
reviewing the product or whether they're actually just
talking with a colleague about an idea to just think
of these three questions and to ask them.
And to make sure that we have really really good answers
to these questions.
So that's why I'm going to talk with you guys about today.
So the first question is the most basic and the question is,
what people problem are we trying to solve?
And the key word here is really the word people.
Because of course whenever we build anything we're trying
to solve the problem.
But what tends to happen is that you start to think
in the mentality of your team or your company and you start
to say things like, the problem we need to solve is that
we need to optimize the click through rate of our page.
And you know, you'll hear things like this all the time,
boil down in small ways and in large ways.
And that's not a people problem.
A people problem as we define it, is if you go out and you
talk to someone on the street and they were to articulate
a problem that they were having, that's how they would
say it, that is the people problem statement.
So there's a couple things that we look at to make sure
that this is a valid people problem statement.
The first is that it needs to be human and straightforward.
So we're not using words like CTR, we're not using words
like optimize or integrate, like these are not words
that people on the street would use, these are not words
that people who are outside of the tech community
are going to use to talk about their problems.
The second thing is we want to make sure
that it's solutions-agnostic.
A lot of times we start problems by saying I am going
to build an app that blank, or I'm going to design
a website that blank.
And already in that statement is an inkling of what
the solution is, but what if you know the app's not
the right way or what if it's not supposed to be a website?
So a good people problem statement gets away from trying
to already constrain it into a particular solution.
The third thing is that it shouldn't be about you know
Facebook or your company or your team or whatever winning.
And I think a classic example is if you say, our service
is going to be the best at blah blah blah.
A person on the street doesn't care if your service
is the one that is the best at that, they just want to know
hey for this problem that I have, what is the best solution?
So stating any problem as we, our team, our company,
whatever wants to win, is not getting close to the things
that people actually want.
The fourth thing is that it gets at the why, so sometimes
you might have a problem like people aren't discovering
this page and that is a problem that we need to solve.
But it doesn't go one layer deeper, it doesn't get at well
why, what's actually the root cause of why people aren't
discovering this page.
Is it too hidden for them, is it too confusing for them
to find, it's gotta get to not just what is happening,
but what is the reason why it's happening.
And finally a good people problem statement can not just
solve functional problems like a confusing flow,
but could also get at emotional or social problems.
Sometimes people just want to feel like they belong,
sometimes people just want to feel like they're validated.
Those are things that also constitute something that
when you ask people they might say.
So let's look at some examples.
So earlier this year I had the privilege of attending
the Mom 2.0 Summit and this is a gathering of a lot
of influential bloggers and journalists who are mothers
and who talk about parenthood and those topics.
And so I got a chance to meet a lot of the attendees
and be on a panel with some of them.
And one of the things that I heard over and over again,
was basically this, and this is an example to me
of a people problem statement.
I want to talk about an interest with other people who are
also interested, but I don't know where to find them.
And this is especially relevant at this conference
because a lot of times what would happen is you're a new
mom and a baby, in your frame group, nobody else is having
babies around the same time as you.
But at the same time there's so many things that goes
through your mind you want to know is this normal,
my baby just did this, like should I be worried,
what are some resources?
You want to have a community of other new moms to be able
to talk to and so a lot of times women will not know
exactly where to find that community and if they're lucky
friends or other people will say hey there's a local
group here or there's a neighborhood mom's group,
but this is something that a lot of people in the audience
were talking about, is something that was a problem to them.
So this is an example of a people problem statement.
Here's another example.
This one we've heard over and over again, ever since
we introduced the Like button on Facebook.
Everybody would ask why don't you guys have a Dislike button
and that would be one of the topmost requests that we've
had for years and years.
But you know that's not really a people problem statement,
that's sort of just a suggestion.
If we dig deeper, the reason why people wanted a Dislike
button is frankly because not everything in Newsfeed
is likable.
People write about hard times that they're going through,
tragedies that are happening in their lives.
Recently we just went through this election which was very
very charged and people would read things or say things
that made them feel a lot of different emotions,
not just happy ones.
And so that is basically the people problem statement.
Not everything in Feed is likable and I want to be able
to easily express other things.
And finally one more example, so sometimes I call these
people problem statements, sometimes they're not just
problems in the way of like hey is someone going to bring
this up as like a hardship in their life that they want
solved, sometimes it's just more of an opportunity
or a thing that if you ask people if they want it,
they would say yes.
An example here is the desire to share spontaneously
and authentically.
A lot of times we're going through a pretty cool experience
and we might capture it at that moment and then an hour
later go and upload it on Facebook but how cool it would be
to actually have that experience live with our friends
and our family.
So that is an example of another people problem statement.
So those are basically the first thing that we always ask,
is okay do we have a statement that is really about what
problem that we're trying to solve for people.
That's the first question.
So now the second question is how do we know
this is a real problem?
And by know, I mean what evidence do we have?
Is there a qualitative evidence that it is,
is there quantitative evidence that we can look at?
And I highlight the word real, not because you know I think
that there are fake problems, but just because I think
the question to ask, is is this a problem
that's worth solving?
All of us have limited time, energy, resources, money,
whatever to be able to devote our being to trying to solve
problems, what makes this one the one that we should pick
out of the thousands or millions of problems
that are out there.
And so this question's really about just making sure
that the opportunity is something that is worth tackling
and that we aren't just solving problems for ourselves
individually but that we actually are very aware
of the problems that the audience that we're building
for is facing.
So for the example of groups, I want to talk about,
I want to find other people who are interested in discussing
the same things as I am.
The way that we try to validate whether or not this was
a big enough use case was of course we went and we talked
to a lot of people but we also looked at some of the data
that we had.
So when we designed the group's product to start with
we imagined that most of the time you would get invited
to the group.
So I might start a book club and I'm going to invite
some of my friends who like reading the same books as me
and that's how we're all going to know about the group.
So in our initial formulation of groups it was really all
about being invited by somebody else who was already a part
of this group.
But one of the things actually that some of the engineers
on the group's team had hacked over the years was this
little unit called groups you should join.
And so some of you guys might have seen it
sometimes you are scrolling through your feed
and notice a little story will pop up.
And based on what your friends have joined and groups
in your community and groups that are similar to things
we think you might be interested in, you'll see
some recommendations and you can go and explore them
and decide to join them if you want.
And we thought that this was just a small feature,
but really most of the inviting was going to happen through
the normal invite process.
But when we took a look at the data what we found was that
actually one third of group memberships were starting
to happen via this Groups You Should Join unit.
Which meant that there actually was a need and a desire
already using the tools that we currently had that
people wanted to find things that they weren't
explicitly being invited to.
That there were examples of all sorts of things that
they wanted to be able to participate in and see
that they just weren't getting connected to any other way.
So this was I think good evidence for us that if we
wanted to devote something, a larger solution to helping
people discover groups, that that was a worthwhile
endeavor.
We also talked to a lot of people in a lot of different
groups, one of the folks we talked to referenced this
group called the Physician Moms group, this is actually
one of the most active groups on Facebook.
And I think something like one out of four doctors
who are mothers in America are part of this group.
And one of my friends actually happens to be a doctor
who just had a baby and so she was lucky enough
to have one of her colleagues add her to this group.
But she talked about how meaningful it was to her and she
would spend hours on this group every single week
because these doctor moms would be sharing tips on how
to balance their work and really tackle a lot of the issues
that come from being in that profession.
And this is an example of a group that everyone who's a part
of it finds it hugely meaningful, but not everyone knows
about it because how would you know that this thing even
existed out in the world unless somebody told you.
Another example is you know I've recently attended our
10 year reunion and at the reunion, a lot of people
were using groups to coordinate events and meetups
and the biggest barrier is that half of the people
I talked to didn't know that these groups existed
that there was a class of 2006 group and that there
was a reunion group and that there were groups
for different dorms who were planning their
get-togethers.
And finally there are a lot of groups, I'm a designer
and one of the top groups that a lot of designers
in the community join is this group called Designer's Guild
and a lot of designers share tools and articles that
they've read and other resources.
And every time I happen to add someone 'cause I thought
about them, that they might enjoy it, they're always
like wow I didn't know about this, that's the first
thing that everyone always says, I didn't know
about this.
So all of this when we did our focus groups in our research
that was more evidence that in fact we should be doing
more to help people discover groups.
So what we ended up doing was actually building a pretty
simple discover feature.
If you go to your Groups tab right next to it, there will
be a tab called Discover and we did our research on what
were the topics that people were most interested in.
So you can scroll through.
You'll see suggestions based on things that you are already
a part of but then you'll also be able to go and browse
by category and this is where you can find parenting
groups, if you're a photographer, people to go on photo
trips with on the weekends or play sports with,
or joining a writing group.
And these are all groups that are public and that actually
want to welcome new members.
For the people problem of not everything that I see
in Newsfeed is likable, and I want to be able to express
other emotions.
We wanted to make sure that this was actually a problem
that many people faced and that the solution that we
had would directly address it.
So we talked to a lot of users, we had them go through
their feeds and describe to us for each story what
was their reaction, what were they feeling, sort of
like free flow tell us what's going through their minds
as they're going through their Feeds.
And you know a lot of times people would say, well
there should be more ways for me to just say something
because what I like about the Like button is that
it's so simple.
I don't have to go and comment and the keyboard comes up
and I have to two handedly type something I like the fact
that I can just in one gesture scroll through things
and say that I like it, but the only thing I can do
is Like.
And there's gotta be other ways for me to express
other emotions.
We also looked at how are people expressing that they
don't like something today and we looked at the stickers
that people were using, leaving us comments as well as
the emoji and what were kind of the most popular ones,
how frequently were people just leaving a simple sticker
or a simple emoji.
And finally we looked at also short comments.
So we looked at how many comments were just one or two words
and all they expressed was like awesome or that sucks
or a very very short phrase that we felt we could
encapsulate.
So with all that, we decided we'd looked at all that data
we'd looked at what were the most common emotions that
people wanted to express and we designed something we felt
fulfilled that criteria of being really really lightweight
so you can still do it with one hand and one gesture,
it's not multiple taps and we also, taking all the data
we had gotten from what were the most common things,
the common reactions that they had.
We built the reactions product and so you just scrub
through and we took the top most what we wanted
to be universal reactions and that's the options that you
see in that tray.
And finally for the people problem of I want to share
spontaneously and authentically in the moment, this was
interesting to us because we had the Facebook Live
product out for celebrities and we built it
for celebrities and public figures first because when we
talked to them that was something that they told us
very directly that they wanted.
And it was something that they were used to because they
lived their lives in the spotlight and they're very
used to going on the red carpet and doing interviews
and being broadcast live.
And it was something they felt they could do to connect
with their fans a little bit more.
So this was something that we had out that was only
available to public figures.
But the interesting insight for us was well is this
something that people want who aren't public figures?
Is this something that people in the audience, you guys,
me, all of us, would this be something that non public
figures would use and find valuable?
And that is the question that we set out to uncover
and prove.
And actually one of the easiest ways for us to do that
was to just build it for ourselves, we already had
the code that ran for public figures and we just turned
it on and see if anybody at the office uses it
and what their reactions were.
And we saw actually a lot of really creative ways
in which people used it, some people used it to broadcast
their team meetings for people who were working from home
a lot of times when RVC programs weren't working
super well.
The other benefit is that you record the video and then
it's available afterwards.
So for people who couldn't tune in live, they still had
a chance to watch the contents of the meeting.
We also saw people just take it out for a weekend
so they'd go to brunch on Saturday with their family
and then they would go live and people would chime in
and there would be this conversation and the people
who did this were like yeah it was awesome,
I had a great time.
And then we also saw people taking it for a run.
So a team at Facebook was doing this relay race
and they wanted other folks to cheer them on and give
them support so they went live and this was also
a super fun thing for them and for all of the people
who participated.
So that was some inkling that hey this could actually
be something that other people wanted.
And so what we did was we launched it to a small
percentage of our users as a test to see if there was
interest in the market.
And that was the Facebook Live product on Android,
it launched on Android first.
And right away we did see a lot of really really creative
uses, one woman who's a tattoo artist started to broadcast
live as she was in the process of tattooing somebody.
We also saw a woman who broadcasted her wedding live
because she had family members who couldn't make it
and who couldn't travel the long distance.
And it was really really cool to be able to see
how she could engage and interact with those audience
members and almost have it feel like they were a part
of that experience.
I had a lot of fun taking it out, I managed to get
reservations to one of the best restaurants
in San Francisco, Lazy Bear and it's a really cool
experience 'cause the chef stands up there and he
explains every single dish and he talks about all
of the love and care that went into how this dish
came together and I just decided to go live and a ton
of my friends tuned in and everybody was like I really
want to go to this restaurant too, so that was really fun.
Okay so that brings us to the last question, so let's say
we built this, we validate it's something that people want
or we think there's enough reason to believe that
it's something that is a real problem and that's something
that is worth solving so now we build it and we put
it out there.
But even actually before the thing is out in the world
in fact even before maybe we have a full prototype
the question for us to ask is how will we know if we
solved this problem?
What would be different in the world?
If we fast forward and now this thing that we've done
is out there, how would we know if we should be happy
if we should be excited, if we think it didn't live up
to our expectations, what exactly
should our expectations be?
And I think this is a really important question
to ask up front, too often what ends up happening
is we have this idea and then we build it and we launch it
into the world and then results come in.
We're looking at the dashboards, we're looking at how many
people are downloading it and what they're saying
and there's tons of data points that we're trying
to interpret and put together.
But it's hard at that point in time to be very objective
about did we solve the problem because a lot of our
natural inclination is to read into the good things that
people are saying and to consider all of that effort to have
been worth it.
And that there's biases that come from when you're already
looking at data and you're trying to interpret whether
or not it's good or bad.
It's much better before you launch to figure out what
constitutes success for you so that you can go
into the launch with that understanding and as results come
in you can map it to your previous, this was my criteria
for whether or not we solved the problem or whether
this was successful.
And so what we do here is we want to make sure we set
measurable goals and metrics and measurable is really
key, measurable doesn't always mean like numbers or data.
But it does mean there's a criteria where if I did this
thing and I got this result I know what to make of that
result.
So for the example of groups, we wanted to help people
solve the problem of helping them find other people
to talk about their interests with.
We ended up building a Groups Discover dashboard.
What does success look like for us?
So before we launched we determined that if we were
successful at actually helping people solve this problem
we would see that more people are then discovering
groups that they're interested in and joining them.
But not just joining because I could make a giant button
flash and make it red and I'm sure more people would
click on it and more people would join, that doesn't
really count right.
What really matters is that they join these groups
and these groups are actually meaningful to them.
Which means that if we fast forward a couple months,
three months let's say and then we look back, these
people are still using those groups and they're actively
engaging, meaning they're talking with other people,
they're reading the content, they're sharing content,
they're liking, they're commenting.
We want to know that people are joining groups that
are actually valuable for them and that they're spending
time on.
So that was actually the more important metric, was not
just joins but meaningful joins.
For the example of I want to be able to quickly express
my feelings about a story but not everything is likable,
we launched Reactions.
And prior to launching Reactions, we wanted to make sure
we measured a couple of things.
If we were successful here then what we should see
is well people are using these reactions and not just
using them, it's not good enough if all the people that were
previously using a sticker or an emoji or a short comment,
then converted to using Reactions, 'cause then it's sort of
a one for one trade.
If Reactions was actually more lightweight, we should see
more people using that then the previous set of people
who were using all of these other tools, so that was
really important to us.
And so that was the first thing that we looked at.
The second thing we looked at was we wanted to make sure
every reaction did get a fair amount of usage across
different markets.
And this was important to us because in the beginning
even when we started to design Reactions, we had considered
lots of different options.
We considered maybe it would be funny if you were
an awesome button or there's tons of different things
that we could've put in order to make it more expressive.
And the reason why we distilled it back down to six
is because we wanted to make sure that it was simple
to use and if you were going through and choosing
from a grid of 20, suddenly it doesn't feel that
lightweight anymore.
It feels like you're pulling up an interface
and you're choosing something and it's going to
take more than a couple of taps to get you what you want.
So it was really important that we nailed and got
to a really small set, but that we pick the right set.
So we wanted to make sure that these were universal and it
wasn't like people in certain countries were using certain
ones more or people in certain demographics were favoring
others because it was a more niche thing.
We wanted to make sure that these were actually things
that expressed emotions people wanted to express in all
of our different countries and all demographics.
And then finally we wanted to make sure that the experience
was good for the receiver.
So if you posted something and then you got a bunch
of reactions, did that make you feel bad did that make
you feel confused, did you not know what was going on,
or did you just feel like it was a negative experience
and it made you want to share less in the future,
that was also really important and that was actually one
of the reasons why we never straight up put a dislike
button because dislike in and of itself can be very
ambiguous.
Am I disliking the content that you posted, am I disliking
you as a person, am I just not agreeing with your point
of view and so in all of our research and understanding
of dislike we knew that it was something that people
were going to get confused by and it wasn't going to be
a great experience.
So when we launched Reactions we similarly wanted to make
sure that that wasn't the case that the emotions that
were being expressed felt like they were adding
to the poster's understanding of what their friends
and audience had to say.
And finally for the example I want to spontaneously
and authentically share in the moment, what we decided
to measure and look at was that of course people were
broadcasting but again it's not just did they broadcast
once because that's something you can make happen by
doing a ton of promotion and putting it front and center.
People will try a lot of things once because they don't know
what it is or they think it sounds exciting.
The true test for us was when they went live and they had
that experience did they like it enough to go and do it
again, now that is the key that tells us we built
something that is valuable and that's worthwhile because
people will come back and do it again and do it again
and they'll incorporate it and make it part of their
lives.
The second thing we looked at is we wanted to make sure
that it'd be fine if broadcasters loved it, everybody
wants to go live all the time but is it really a good
product if nobody wants to watch what people
are broadcasting live and then it's one-sided
and over time broadcasters are not going to want to do
'cause nobody ever watches anything.
We wanted to make sure that the content was actually
interesting and the way that we can measure that
is the time spent watching live videos going up
proportionally.
And when we run surveys and other things are people saying
that this is a valuable thing that they want to watch.
So to recap I went over the three questions that we always
ask in product reviews and you know anytime we're talking
about products.
We want to make sure what people problem we're trying
to solve that it's very clearly defined, that we can picture
the audience or a particular person in mind as we're
designing the product and that we're not solving problems
for ourselves or our company or our team but that we're
actually focused on an audience and a problem that
matters for them.
The second thing is just making sure we know that it's
a real problem through looking at whatever quantitative
data we have through doing focus groups and talking
to different users, understanding that this is something
that is of good use of time for us to work on
and to solve.
And finally being very rigorous about what does success
look like and how will we know if we put something
out there and it did what we wanted it to do.
And if not, then we go back to the drawing board,
we learn what didn't work and we keep iterating.
But these three questions have helped keep our teams
focused on what matters which is ultimately
the people that we design for.
Thank you guys, thank you very much.
(audience applause)