It's great to understand the numbers because the metrics are the foundation
that you need in order to make the right decisions for your business.
However, sometimes even the numbers can be worthless, if you can't really
explain why certain metrics are moving, or why things are happening.
This is when you want to hear from your users to get some
qualitative answers and reasons.
UserTesting.com is a tool that can give you a wealth of information and
insights into what users are feeling when they use your product.
Let's hear from Mike, VP of [INAUDIBLE] UserTesting.com about UserTesting.
>> So, why do you want to see people actually using your app?
Analytics can tell you wonderful things, and
it's really important to have the best analytics you can possible get.
But what they tell you is,
what people are doing and you don't necessarily learn why.
So you may see a problem but you don't understand why it's happening.
And then you end up learning a whole bunch of different experiments to try to
figure out what's going on.
And they won't necessarily get you really quickly to the right answer.
Where as if you can combine the analytics that tell you what people
are doing with the video that tells you why and what they're thinking and
stuff like that, you can much more quickly get really, really good overall
insight that kind of combine both qualitative and quantitative measures.
So it's a way to learn faster.
So one example of this sorts of things that people learn from testing
the listings for the store is even the art style
of the icons can be really, really important.
So it's not just what you show, but it's are you doing for instance,
a cartoony-type version of the icon, in which case,
that kind of suggests a certain type of audience.
And sometimes you'll find I don't want to be sexist about this at all, but
sometimes what we see is the women sometimes respond better to something so
little more cartoon looking because it feels a little bit more friendly.
Think of the sort of graphics that you see in a game like Candy Crush which is
really aimed at women.
Then sometimes you get more hard-edged graphics that with the little more
realistic looking and maybe you've got if you got face frowning face that looks
aggressive or something like that.
Sometimes again, it sounds sexist, but the guys respond better to that sort of
thing and so it's not just what you show but it's also how you present it.
The graphical style that you use, the artistic style,
all that sort of stuff you need be very, very conscious of.
Combining analytics and user tests is one of the best things,
one of the most fun things that you can do.
Because the analytics will give you all sorts of information on what's happening
with your application.
And you'll inevitably identify some places where people are getting stuck.
Oh they get to the third panel and
they cant figure out what their, why are they not clicking on the button?
It's so freaking obvious to the design team.
Right, because we worked on it for nine months now and
so it's completely intuitive to us.
They're not tapping on that same thing and
you'll get disputes within the team sometimes.
Religious disputes about well, it's the color.
No, no, no, it's the design of the icon.
And you could go and run like 15 different AB tests to try to figure out
all that stuff, and spend an enormous amount of time.
And what we recommend is instead we reach that point.
One of your user tests and it doesn't have to be 50, okay.
You're not looking for quantitative stuff, this is qualitative,
it could be, maybe five.
Just to get some reactions from people, give them instructions,
go to the screen and then tell me what you're going to do next and
talk to me about what you're thinking.
So don't just watch what they do but you can prompt them with,
I want you to tell me when you look at the screen what are you seeing.
And what do you choose to do next?
Why do you choose to do that?
And usually what you'll find is they're getting confused or their getting
distracted by something and half the time it'll be what somebody was arguing
within the team and half the time it'll be something that nobody realized.
Like oh my, oh my goodness that annoyed them, or
that made them feel like they weren't well connected to it and [SOUND], okay.
But the good news is that you found out quickly.
So you can then for your AB test that's going to follow,
you're B version can be affixed to that particular problem.
And so with luck maybe he'll only have to do one AB test or maybe you'll only
have to do a couple of rounds in order to get it to really really work better.
So what it does it is makes your analytics in your AB testing
more effective it let's you move through that stuff faster.
So you evolve the app faster, you get better quicker and therefore,
you are able to reach your ramp point a lot faster.
So other things you can do with user tests, the cool thing is
you're not just limited to recording what's on the screen and
get the end users voice, you can also turn on the backside camera.
So something that we've done for some developers is
If you've designed an app where you want it to be taken out into the real world
you can get some context about how it's being used in those places.
To give you an example we did some stuff for one of the major real estate apps
and they assumed that the app was going to be used at for
home browsing when they're at people are out looking at homes.
And what they found instead was that people were using it at home on
the couch to plan out where they wanted to go.
And so, all of a sudden, you realize the context of the usage
of the application is different, so I'm competing with the television.
I'm competing with stuff that's playing on the stereo, but
that also means different ways you can potentially market to those people.
If they're using it at home while they're watching TV,
oh, what shows do our target users go after.
Are there some ways we could look for placement, or things like that,
and obviously, if you're a small start up maybe you don't have the money to
look at those sorts of things, but you can still look rather than advertising.
Maybe it's placement opportunities or how do I influence the people
that I know that they're watching or things like that.
So the more context you can get around the life of your user in
the situation in which your stuff is being used.
The better you're going to be able to
be able to tailor your stuff exactly to what those people are going to want.
I think another thing is, you can use it also just to kind of interview them.
So obviously, you should be talking to your customers all the time,
you should be meeting with them.
You should be taking them out to coffee or doing whatever.
But there may be situations where they're geographically diverse and
you want to see what are people thinking in other areas.
You can do a user test that just asks them questions and asks them to talk.
You don't have to have them using your software.
It can just be a way to record a quick interview with these people.
Think of it as focus groups, where you don't have to rent a focus
group facility, you don't have to schedule these people to come in,
you don't have to pay them a whole bunch of money to come in.
And then by the way, half of them don't show up and you sit there eating m and
m's while you're waiting for somebody to come out.
It can be a way to just get some dialogue going with your users and
get them talking to you about what they're thinking.
So one more interesting use of these sorts of user tests to be aware of
is that the videos themselves can be very, very persuasive.
So, you may be in a situation as a marketing person where you've gotten
an insight about how the customers think, but
maybe other people in the company have their own ideas.
And you can do the traditional thing, which is you go have a big long argument
with them over a whole bunch of different times to gradually move them
over, or what you can do is just forward the videos around.
And in fact, one small developer I was talking with the other day,
they're a company called Trove, that does personal property management for
kind of like insurance purposes and stuff like that.
And so they've got like a 15 or 20 person staff, and they said, well,
we get together and we do viewing parties in the videos and
sometimes they're great because they confirm things that we already thought.
And sometimes they're really, really painful, but the nice thing is,
nobody argues with a use who's saying what they actually think.
And you know that these tests are being really effective when you get other
members of the team, like engineers or people like that, coming back and
saying, I saw this in the video, and I'm starting to think this is the direction
we should go, because then you're just educating people.
You're not having to argue, you're not having to create friction.
It's not taking a long time, but naturally,
the team's coming together around these insights.
And to me that's one of the best uses of these tests.