KORY JEFFREY: Welcome, everybody.
We are on site at Founder World in San Francisco.
I'm joined in the studio right now by Michael.
He is the VP of Mobile at UserTesting.
Thanks for joining me, Michael.
MICHAEL MACE: My pleasure.
KORY JEFFREY: So let's start with what UserTesting is all
about.
MICHAEL MACE: We are a really, really, really, really,
fast way to get user feedback on an app or website.
You come to our site, you define the demographics
you need, the questions you want people to answer, the tasks
that you want them to perform.
And if it's an app or website, you upload your build to us
or give us the URL for the place.
We then deploy it out to our panel.
They video themselves doing what you've
asked for, so you get audio and video of the screen
and of them talking.
It's on a mobile device.
We capture taps and swipes and stuff like that.
And then it gets sent back to you so you can watch it.
The turnaround time can be as low as an hour.
KORY JEFFREY: Wow.
MICHAEL MACE: So it lets you get user feedback within a design
sprint.
So you can do a new build in the morning,
you can then send it out over lunch time
and by the time you get back from lunch,
you're starting to get those videos.
So really, really fast feedback on usability
issues, or do they understand stuff,
or other things like that.
KORY JEFFREY: Those videos-- are they bite size?
Is there any way to avoid data overload?
MICHAEL MACE: So if you want, you can do it all yourself.
You come and define the test and you watch all the videos,
and that's fine.
If you want, we can do all that for you.
So we'll watch the videos, we'll pick out the highlights,
and we'll explain-- OK here's the problem.
Here are three little 30 second clips.
So it's basically whichever way you want to use the service.
It's more expensive if you have it full service,
it's less expensive if it's self service.
KORY JEFFREY: So how does that scale for you guys?
Obviously having to, within an hour or two,
be able to turn that around?
You would need a lot of testers.
Where do you source those?
How do you do that?
MICHAEL MACE: A lot of it has been through part-time work
boards and other sorts of places where we get word of mouth.
So we haven't had to advertise very much.
It's kind of built up itself over time.
The company's about eight years old,
so we've had the luxury of being able to grow it organically
over time.
And we've got about a million respondents available now,
so it's turned into a pretty big thing.
We do thousands of tests per week.
KORY JEFFREY: Based mostly in the US or all around the world?
MICHAEL MACE: English speaking countries.
So US, UK-- we're starting to expand other places.
We can do testing in India, for instance.
So if somebody needs a particular country,
we can usually do some custom recruiting for that.
KORY JEFFREY: Having run eight years' worth of tests
and seeing all of this stuff, you must see a lot
of interesting tidbits that would be really helpful
for entrepreneurs as they think about this.
Can you share some of those?
MICHAEL MACE: Sure.
A couple of greatest hits, I guess.
One would be-- never assume that an icon or a particular word
is going to be intuitive to your users
just because it's intuitive to your team.
You have no idea how many weird icon designs we've
seen on buttons where the designers think,
oh it'll be obvious that you push this in order to do that.
And when the users get to it, it's
just like, what in the world am I supposed to do here?
KORY JEFFREY: OK.
MICHAEL MACE: So my personal recommendation--
I know it's not pretty, but I think the only two icons that
are universally recognized on mobile devices
are the little magnifying glass means search,
and the question mark means help.
Everything else, including the hamburger menu, a lot of people
will misunderstand.
And so maybe you put words underneath.
Even though it's not as pretty, it
makes your app look a little more cluttered.
The question is, are you shooting for pretty
or are you shooting for usable?
When you have to choose, I think you want to go for usable.
The other thing we've been seeing a lot lately,
that I think is a really important one,
is we are now able to test out in the wild.
So just recently we've added the capability.
We can deploy these tests out directly to a smartphone,
for instance, where somebody who was walking down the street
can use your app.
The level of distraction in the real world when people
are using mobile because they've got stuff going on,
and they're getting social notifications and stuff
like that-- it's unbelievably high.
So you'll see even a beautifully designed app that is well
designed to convey information.
People will see stuff that is written right in front of them
on the screen, and they just won't see it.
It's this cognitive distraction thing
where they've got so much input coming in when they're out
on the street that your threshold for being obvious,
being intuitive, being usable, is much,
much higher in the real world than it is when you're doing
a design review in some quiet room
where you project it up on the wall and stuff like that.
That's not reality.
You need to be testing stuff out in the real world.
KORY JEFFREY: Interesting.
So how do you get past that?
What are a few little tidbits to get
past the design issue of having so much information going in?
MICHAEL MACE: One thing is the self-serving thing
of test stuff on UserTesting-- we'll set that aside.
KORY JEFFREY: That's good to start with that.
MICHAEL MACE: I think the big thing
is, being really sympathetic to the situation that users
can be in and back stop them.
So for instance, we tested out one of the major restaurant
finding sites to see, OK, let's have somebody actually pick out
a restaurant and walk to that restaurant.
Let's watch their whole progress as they walk down the street
and all that sort of stuff.
We can turn on the back side camera.
We watch them as they walk down the street.
And what we found was, one of the testers
ended up walking to a restaurant that was closed.
And when we looked back at the app,
it didn't notify them that restaurant was closed.
But it was just in one spot on one of the panels,
and it was just one word.
And they flipped past it because they were in a hurry.
So if you're designing that app, maybe what you want to do
is say, OK-- they went from the app listing
to going to the map function within the app.
So they're obviously walking to the restaurant.
The restaurant is closed.
Maybe we want to pop up a dialog that says hey, just to remind
you, the restaurant's closed.
Are you sure you want to walk to it right now?
So you should assume that people actually
don't see notifications the first time.
And if, by monitoring their behavior,
that tells you that they probably
didn't get it the first time.
You need to repeat it several times.
Sometimes that's going to pester your users a little bit,
so give them the chance to opt out of that if you're
getting to be too annoying.
But saving somebody 10 minutes when they're
walking down the street?
That's a huge deal for a mobile app.
KORY JEFFREY: In that instance, we
talked about using the back side camera and access to location
and things like that.
Do you deal a lot with people and permission flows
on how they ask people what they can use and when,
so that they're able to do these types of things?
MICHAEL MACE: We get permission ahead of time
so the users understand exactly what's going on.
We add a little red bar at the top of the screen
when we're recording so that they're reminded
that recording is going on.
And we also instruct them on the things that they can't do.
The big thing is, you're not allowed to interview somebody
else without permission.
So you can record yourself and that's just fine.
But for instance, if I had it running here
and I was pointing the camera at you,
I would have to ask you permission under US law.
Otherwise that's some horrible wiretapping thing or something
like that.
It's awful, you don't want to go there.
KORY JEFFREY: I guess the example I want to touch on
is-- if I'm building an app, right?
And I have to ask for permissions for things?
Do you ever come to a point where
you have to actually design for when you only
get partial permissions?
MICHAEL MACE: Generally what we do with the testers is we
get their permissions up front.
If you're asking for special permission within the OS,
like your app is asking for their location.
That permission dialogue is going
to pop up the same as it would any time other groups are
using that app.
So they'll just do that.
But for permission to record their voice,
to record their image, take information from their device?
We've already cleared all that stuff with them.
KORY JEFFREY: Interesting.
OK, so we're at Founder World.
There's been thousands of people walking here
over the past couple days.
There's a lot of exciting technology on the floor.
What is it that you've seen that's excited you here?
MICHAEL MACE: In general the thing
that's got me the most excited, not just here
but in the industry in general, is all the VR
and 3D stuff that's going on.
I guess it's an advantage of being
in the industry for a while.
I've seen some of the former platforms
since they became really prominent
like the graphical interface and the Web itself.
The VR stuff-- when you combine 3D printing with all the VR
visualization we're able to do now-- it
feels like the early days of those platforms.
As we dig in, we're starting to be
able to create some really surprising non-intuitive stuff.
I had no idea.
We don't know yet what to do with the technology.
To me, that's the most exciting thing because these developers
are literally creating the future right
now If I weren't working at UserTesting,
I'd be going digging into that stuff
right now because I think it has huge potential.
KORY JEFFREY: Really cool.
So a relevant, but maybe side track question--
do you have a favorite Google technology?
MICHAEL MACE: It's hard to pick.
I don't want to offend any of the teams,
but since I'm a mobile guy-- Android.
KORY JEFFREY: Android?
MICHAEL MACE: Yeah.
Because it's done so much and has so much
impact around the world.
KORY JEFFREY: Thank you very much for joining us.
MICHAEL MACE: My pleasure, thank you.
KORY JEFFREY: Lots of fun.
And thank you guys for joining us.
We are on site at Founder World in San Francisco.
[MUSIC PLAYING]