PAUL: Welcome to this session on Performance Culture.
I am Paul, on the left there.
I work on Google's Developer Relations team
where I spend my time looking at performance, design, and UX
and normally where the three of those meet.
LARA: And I am Lara Swanson.
I work at Etsy and I'm the Engineering Manager
for the Performance team.
We help all of the feature teams who
are building products or features or experiments,
make sure that whatever they're shipping
is as fast as humanly possible.
PAUL: Alright so, I guess the opening question is, why are we
even talking about performance cultures?
Why does this session even exist?
And I think to answer that we need
to understand a little bit about where our industry is
and what we are seeing and where we think it's going to go.
And it's really about this.
The multi-device web.
We've seen more devices than ever before come online
and I think really the one that stands out
for me the most is this one.
It's the smartphone.
It represents the most performance constrained device
that we have.
Whether that is in terms of the screen size,
or the GPU, the CPU, or its connection to the network.
And the thing is more and more people are using smartphones.
In fact, one study suggests that just over a third
of American adults use their smartphone
as the primary means of accessing the internet.
Which means for those people their first experience
of you, your site, your brand is going
to be through their smartphone.
And that is something that we need to be thinking through.
In fact, it's not just in the States it's also globally.
So, here we have the percentage of total global internet
traffic according to Stat Counter
and you can see that the trend is pretty clear upwards.
And it looks I think sort of the middle of next year,
if the trend line is correct, that well
cross that 50% marker.
So the thing about this is--
LARA: Mobile networks can add a tremendous amount of latency.
So, before a mobile device can transmit or receive data
it has to establish a radio channel with the network.
And this can take several seconds over a 3G connection.
After the device talks to a radio tower
to negotiate when it can transmit data,
the network carrier then must transmit that data
to its internal network, and then to the public internet.
So, the combination of these steps
can add up to tens of thousands of milliseconds
of extra latency.
So, the most important question I
hear when I say this is, well what about 4G?
4G is awesome right?
Our networks are improving.
But you have to remember the new device and fast network
with an excellent connection that you have in your pocket
right now is not a good representation of what your end
users are really experiencing.
So, while it's true that networks are getting better
you need to remember that your users are
on a variety of connections, with various levels
of connectivity, and latency.
It's also important to remember that if you're a site is slow
people will go elsewhere.
In fact, one study suggests that as many as 40% of people
will leave a site when it takes longer than three seconds
to load.
So, we have some options.
In the face of the stuff that's a little bit
scary what can we do about it?
The first option is to ignore it.
You can certainly say, you know what we just
saw those awesome stats.
We say that we're actually-- we're trying,
but we don't think it's that important.
Maybe mobile isn't the future, maybe that trend line is going
to lie to us, maybe networks will be better immediately,
I don't think that this is true.
PAUL: No, I tend to think-- no.
LARA: I think that we can't just ignore it.
PAUL: I'd like to.
LARA: Yeah, it'd be really cool, but I
think the other option is to assign performance cops.
Right?
Who in here would identified as a performance cop or janitor
within their organization?
PAUL: Anybody ever been one of those?
LARA: Yeah, somebody who comes in afterwards cleans up.
PAUL: Look at these hands.
Look at this. [DEEP VOICE] Yes that's me, I don't know.
LARA: So, performance cops, right?
We come in and we say, hey designer, hey developer here's
some better ways you can do this.
You come in cleaning up after people,
you try to make your site better,
but the responsibility solely rests on you.
And this can often lead to a tremendous amount of burnout
and frankly, it's not sustainable.
There's always going to be new people joining
your organization, your site will continue to get slower,
it'll continue to be iterated upon,
the hardware age, being a performance cop
is not a sustainable thing, so that's
why we're here to talk about building performance cultures.
PAUL: Wow, I wonder which one we'll choose from that list?
[LAUGH]
PAUL: So, we're all set to build a performance culture, super.
But we have to ask the question what
do we even mean by a culture.
Well I guess you could come up with your own definitions
goodness knows there's probably a bunch of them
that you could think up, but for us today,
at least in this conversation, here's the things
that we actually have in mind.
Firstly, it's a way of saying, I belong to this group,
I get them, hopefully they get me,
I understand this particular group.
So, for example being British, I would
identify with British culture.
LARA: You're British?
I know right it's a relief for anybody in the room going,
where is he from?
His American accent is really strange.
LARA: [LAUGH]
Yeah, I know.
So, it's that first and foremost is just saying,
yep this is my kind of people.
It's also about how you think and how
you reason and rationalize the world around you
and the kind of social cues that you
look for or the slightly strange obsession with green liquids
you may or may not have, but it's
that sense of the world around you.
It's also how you do things.
Whether that's the side of the road
you drive on maybe the wrong side, the words
that you use for things or perhaps
in our case how we go about crafting the code.
And then lastly, it's how you celebrate things,
the things that are important to you,
the things that you celebrate and how
you go about celebrating it is unique to your culture.
And we see cultures at the highest level
internationally, nationally, all the way down to our homes
and so forth, and actually of course, our workplaces,
which brings us back to that whole performance culture
thing.
LARA: Yeah, we were joking when we
were talking about earlier that performance cultures are
kind of a team sport.
It's important to remember that everybody has to play.
And as you say about sports there is no "I" in performance.
PAUL: That is true, but if you're willing to look hard
enough you'll find there is "prance-for-me".
You can stop the slide now.
LARA: Alright, alright.
So, thank you for that.
So, culture change is scary and it's hard
and I can understand especially for those of us in this room,
we don't know necessarily how to approach it.
How do we start to create or enact performance culture
within our environments?
I'm going to go through some real things that
have been said to me as I've gone
on this journey within the organizations I've worked in.
Lara, "I don't want to think about mobile."
Right.
PAUL: So whiney!
LARA: Yeah.
"Lara, I'm a project manager or a product manager
and I've look at the deadlines, I've got some resources
and frankly building for both desktop and mobile is going
to take twice as long." [WHISPER] It's a lie.
"Lara, you at the performance Manager,
don't you know responsive web design is
inherently bad for performance?"
Or my personal favorite, "everything changes too fast!"
So, here's the thing we've been through this before
and when it comes to web development
we've gone through these kinds of change the past.
Who here has built sites with tables for layout?
Yes.
PAUL: Yes!
Look at all these guilty hands.
[DEEP VOICE] I don't do it any more.
LARA: It was a pain.
So, at probably every job I worked at between 2006
and 2010 I remember these arguments,
do we have to move to a CSS based layout?
In fact, we did.
We did it pretty successfully.
This is more like what we would use today,
as is actually said to change again with web components.
Who has heard this from a very important person, excuse me,
the important part is not above the fold.
When the EP's would have to scroll to see content.
Yeah.
This is again what we're seeing today.
We used to worry about 800 by 600 pixel monitors,
now we're worrying about the spectrum of screen sizes.
This graphic represents all the devices
that come to ETSY.com that have more than 1,000,000 visits
per month in traffic.
We're no longer seeing just handsets and tablets
and desktops we're seeing a spectrum of screen sizes.
You may also think about how we've
changed developing for different browsers over time.
I remember back in the day we would talk about cross browser
compatibility or making things look the same across browsers.
Browser hacks, prefixes, et cetera and this
is again what we're seeing today.
We have to start thinking about mobile web browsers
in addition to just our desktop clients.
But this is all to say that change is scary,
but it's also constant in our industry and it's good.
This is a good part of our jobs.
We've lost crazy table layouts and now we
have semantic markup and more people than ever
before can use our sights on our apps.
This is good for us as developers
and it's really good for users.
PAUL: So, we need to embrace change, good, fine.
So, where do we start when we want
to create one of these performance cultures?
Well, the first step for us is to actually gather some data
because essentially you don't know where you're
going if you don't know where you are
and you need to understand how real people use
your site, like what devices are they on?
Where are they in the world?
What journey did they take through your site?
Which pages did they arrive on and which ones
did they leave on?
You need to understand your own situation
and if you don't know today you can't
start telling that story to the people around you
and get them on board to build this culture with you.
So, the immediate thing that springs to mind
is just going and have a look at your analytics.
Go and look at Google Analytics.
Figure out what it's telling you.
Spend a bit of time, make reports,
and start to get that sense of where you're at.
The next thing you want to do is actually
look at you load stats.
So, how long does it take to load it?
How long does it feel like it takes to load your site?
Again this is coming down to the data that you can share,
but it's also in this case coming down
to a little bit of empathy as well.
How does it actually feel to be one of those people that's
using your site on a day to day basis
from somewhere else in the world.
And for that we can turn to Web Page Test
and you can ask it to load your site from a range of locations,
a bunch of devices, and different connection types.
And it is going to give you a load of really useful, really
amazing data.
And one feature I particularly like
is the ability to compare pages, so you could take for example
your own site and oh, I don't know,
the site of your co-presenter and see who's fastest.
LARA: Not cool bro.
PAUL: Oh yeah.
I actually did that, as you can see
and depressingly it's a draw.
So, yeah.
Could have been bad, but it wasn't.
I'm sure you can start to see how
you could start figuring out the story here.
Like how does it feel to be one of our users
when you compare it to somebody else in the world or-- da-duh,
da-duh, da-duh you get this idea.
OK.
So, you start to get in this insight
and you don't even have to leave your desk.
That's the best bit.
But we can get even more data at Web Page Test
and we can share that with our team.
Like how long does it take before our server sends
some data back or how long before our pages
start rendering?
But one number I want to call out
is this, which is the speed index.
It's a bit weird the way it's calculated.
You can click on there there's a link.
And you can click on it and it will tell you,
in all the gory detail, how it figures it out
and it's pretty cool.
LARA: It's magic.
PAUL: This is magic.
But it is a really good number for figuring out
in milliseconds how long it feels
like it takes for your page to load.
So, that's a really good measurement,
better than say, the onload event.
But speaking of perceived performance,
we don't just want to stop caring
when people-- when they've landed on the site
and we go oh, what have we done.
We need to care about scrolling and animations and interactions
and they all need to be this silky smooth 60 frames
a second because people care about that.
They want that feel of responsiveness and smoothness.
Again you go to like Dev Tools to look at your pages for data
and figure out what your frames per second is like.
So we get frames per second, we can
dive into the frames a little bit
there and find out where we've got issues
and then start fixing them.
Now if all that's new to you it's
time for a shameless plug, which is
to say I've got a session tomorrow morning that will tell
you exactly what you need to know about these tools,
"The Tools You Need to Succeed."
It rhymes, it's true.
So, come to that if you're interested.
Anyway, hopefully at the end of this situation
we have a load of information, we have a lot of data,
and we're starting to hopefully build this story in our heads
of what it feels like to be one of our users.
LARA: So, you've got this data and you're excited about it.
What do you do with all of that excitement
and all of that data?
You should go talk to some very important people.
You want to enlist the help of the very important people
at your company and get them to understand what your users are
feeling and experiencing.
There are few things more powerful
than a very important person saying, this is important.
Performance is important.
It's crucial to getting others at your company invested
in performance.
So, what can you do to make this happen?
Here's an idea that we came up with,
you should send your very important person one stat a day
for two weeks and see how long it takes before they cave
or just get really annoyed with you.
One that we often like to cite is this Google study
that found an additional 200 milliseconds of delay
triggered a 0.3% loss of engagement.
You may say to yourself, 0.3% is not a lot,
but Google is pretty popular.
So any kind of loss of engagement
is going to have a huge effect.
Or remember that old stat that we just had where 40% of people
will leave a site that takes longer than 3 seconds to load,
tell your very important person that stat and say, by the way
our takes 5 seconds to load on a 3G connection.
So, after the stat thing has run its course
consider running some experiments.
Measure the business impact of these experiments.
Know what kinds of metrics your very important person
cares about.
Maybe it's balance rate, maybe it's conversion rate,
maybe it's something else.
Go ahead and run performance experiments
and see what the effects are.
On Etsy, we ran an experiment in which
we added 160 kilobytes of page weight
to a page on a mobile device and we
saw that it increased mobile bounce rates by 12%.
This is the kind of thing that we
can share with the folks at Etsy and say, hey,
this stuff is important look, at the effects that it has.
However, this is kind of a negative stat
and I kind of prefer to make a positive.
So, what we found when we eliminated jank,
that's that scrolling issue, when
we eliminated jank on Etsy's activity
feed we saw people "favorite" more often,
and "favorite" more items.
And this is another kind of experiment
that you could run, make something better and measure
the effects before and after and pick and choose
the engagement metrics that you know
your very important person is going to care about.
You could also use that web page test film strip, thanks
for that--
PAUL: Your welcome.
LARA: --and take the video output
and compare that before and the after the experiment,
make an improvement, show how slow it was before
and show how long-- how short it took afterwards.
And get that person to care and feel about how this works.
Or compare your site on desktop to how it loads on mobile
or compare you're site in the United States
to how it loads globally.
Get your very important person to see and to feel these things
so that they actually care about this stuff too.
You want to get them to feel it.
Once they understand you can help them empower others.
By changing how you're very important people think
about performance it will help everybody who's surrounds you.
PM's, designers, developers care and work on performance.
PAUL: Speaking about those other people, so, we've got data,
we've got our VIP's to say, OK performance is important,
we see that it's got real value.
How do we then make it so that the people around us
can actually-- start actually taking this and using it.
Well, the easiest way is to educate them through something
like a brown bag, lunch and learn session
on all the tools, all the data, all the things
that you've figured out about how you're actually doing.
And it's not-- as we said it's not just
developers, it's designers, is UX,
it's project managers, it's anyone
who's involved in what you do because it's a team sport.
And we want to make sure that the people that we're
working with know the impact of the decisions that they take,
so it's not the case of, well the developers
have done all they can, but actually the problem
is originated from another place.
You know if you include everybody
you've got a much better chance of success.
The next thing that's well worth doing
is to start documenting your best practices
and start writing things up.
Because what gives you is an easily copy,
pasteable set up that you have.
It makes it more obvious about how
to do the right things as far as you're concerned.
Anybody who joins your company is going to start figuring out,
oh this is how I'm supposed to do things,
rather than just having to guess and muddle their way through.
And you can use it to show what worked and what didn't work
and if nothing else it kind of gives everybody a focal point
and say, right this is-- because you will find there will be
trade offs in everything you do.
This thing and doing it this way works because of this.
So it's a really good place to sort
of-- it's really good to start writing these things down.
Another thing that is definitely worth doing
is seeing what you're uses are seeing,
which is to say invest in a device lab.
And I mentioned this to somebody the other day
and they went, "psha $10,000, really?"
Because that's what they were assuming it was going to cost.
Well first of all it doesn't necessarily
have to cost that much.
Secondarily, the cost of not testing across multiple devices
may be way higher and so, that's something
that you want to decide for your selves.
But it's a sort of-- it's an out of sight
out of mind kind of problem, this one.
If you put these devices in front of everybody
it kind of gives everyone that reminded
that there's this whole range of devices they should be testing
on and that they are building for the multi-device web
and that they need to care about performance on all of them.
It's also a social place as well,
so people can meet and gather around these devices
and start to figure out together how you can solve
particular problems and again that will broaden
the horizons beyond just the developers.
One thing that is pretty cool out in the Design Sandbox,
if you haven't seen it we have a Device Lab.
And the team there will talk to you
about how you can go about making your own device lab.
The source apparently is on GitHub
because it does some crazy stuff where
it which is to all the devices at the same time, which
is really cool.
So go and see the Design Sandbox people they will help you.
LARA: And then also if you go to laraswanson.com/devicelab you
can see all the slides that I just gave for a tutorial on how
to build your own device lab.
How to choose devices, how to troubleshoot power, and all
of the other things that we've learned along the way.
PAUL: Awesome.
Alright, so, you-- it's great to have the device lab,
but it's not great if you don't use it.
It's a bit like having a gym membership and they never
getting on the treadmill.
Yeah.
So, make time to actually test your stuff for real.
It's good you've hopefully got the VIP's
to say it's important so you can say actually we need time now
to make sure that things are working well.
So go and give your devices a big old hug.
Let them know you love them.
Like we said earlier though educating your team
isn't just about developers and performance
isn't just about an internal thing some of it--
sorry it's just not an external thing it's also internal,
get that the right way around.
It's about everyone who builds for the web in your company.
And that means in a lot of ways that you
want to eliminate duplicate code,
you want to basically embrace responsive.
Instead of having a mobile team that does the M-dots
and you know the other team that does all the other stuff.
M-dot.
What do you do for TV?
Do you do TV-dot?
LARA: What do you do for refrigerators?
PAUL: F-dot?
LARA: R-dot?
PAUL: R-dot.
LARA: I don't know.
Don't do that.
PAUL: No, don't do that.
That seems like it's not going to scale well.
LARA: No.
PAUL: OK, you said earlier they do taxies.
LARA: Oh yeah, taxis in New York have the little--
you can access the internet.
Mind, [SHOOTING SOUND] blown.
Alright, point here is your going
to eliminate duplicate code, you going
to make it easier for your team to iterate more
quickly, hopefully, and start to break down
the barriers between you, the designers, the developers,
and it just gives you a much greater chance of success.
So, in summary, when you've got the green light from your VIP
start teaching your team about performance
and the impact of their work, it's going to empower them.
Sorted.
LARA: So, you need to start surfacing performance
within People's daily lives.
We can talk about performance all day
about how important it is and when you ship something new you
can see the end of the day how much performance was impacted
negatively or positively.
But this isn't help people as they're
designing or developing.
So, the first thing you can do is add performance to your
build tools.
Automate image compression.
Get a command line tool running like Grunt or Gulp that
runs performance tests and compares
your new work to your performance budget.
You could use something like Travis
to do the in a continuous integration environment.
And these tools can even report back
to dashboards, which is my next recommendation.
So, at Etsy we look at something like this every time we deploy
code, which happens upwards of 50 times a day.
When someone deploys something new
they'll go over to a dashboard and say,
how did this impact performance?
It's really important that as people are continually
pushing out new work they can see immediately
how their stuff has affected it.
But also at Etsy we have this tool bar
that sits at the top of every page of etsy.com
if you logged in as an Etsy employee.
And in this we've got information
about visitor traffic and experiments,
but we also have performance data.
So, we surface the timing's for people
as they're developing in developing, staging,
and production environments.
And we also have a clear baseline
for what is acceptable.
We'll let you know if you're violating
one of our performance service level agreements.
By surfacing your team's performance data
during the workflow it'll help improve their work,
it'll become baked into their workflow.
They'll start to care about it because they
can see how their work directly impacts it
as they're designing and developing.
And this brings me to our last point.
This stuff is going to make people so excited
and people are going to ship things
that are improvements to your site.
You should be celebrating everything.
Again, I'm all about the excitement.
I'm all about getting people pumped up about performance.
PAUL: Prepping the talk with Lara
was like an object lesson in excitement.
LARA: Nice.
So, for me, this is actually the most important part.
Back in the day in August of 2011
the man behind the sunglasses, Seth Walker,
published Etsy's first performance report.
He published it on Etsy's news blog, so it's super public.
He chose a bunch of baseline business metrics or page load
time metrics, rather and he showed
how long are our major pages were taking to load
and you can see there's a huge outlier here, the homepage.
It was embarrassing, but he published it anyway
because this stuff isn't a secret.
What's amazing about this is that developers teamed up
to fix this.
As soon as we publish it, as soon
as Seth put that into the world people got excited about it
and they wanted to help and they want to fix it.
And by the next time we published reports, which
was in November of 2011 there was a huge improvement.
Not only did the home page significantly improve
in page load time.
It was the fastest one on this list.
So, you should be sharing your findings.
Make sure that you're celebrating
externally the stuff that you're doing to help improve things
along the way, but also as Paul mentioned
it's important to do this internally as well.
This is an example dashboard that we have,
the Performance Hero a dashboard,
in which we celebrate engineers and designers who
make huge improvements to Etsy's page load time
and they're not on the performance team.
In this case Chris Fairbanks, who's on the checkout team,
eliminated a ton of duplicate code
and had what we call a Baumgartner.
Remember that guy who jumped out of the hot air balloon
and landed on earth?
Yeah so every time we have a graph like this--
PAUL: It wasn't a dance move, right?
LARA: It wasn't-- no, it wasn't a dance move it was a plummet
to earth.
Every time we have one of these, a Baumgartner,
we celebrate this internally.
We put this up on a dashboard and we
have that little printout sign, and we hang out with them
and we're like, thanks.
Thanks buddy.
And we want to celebrate the things internally
lots of high fives, lots of doughnuts, lots of cake.
You should be rewarding your innovators.
It's not just about coming down with a hammer.
It's not about being a cop or a janitor.
This is really about celebrating improvements and getting people
excited and invested in their performance.
Celebrating wins will motivate your team
to be performance champions.
So, to summarize gather the data.
You want to make sure that you have
a baseline of understanding how your users feel.
Again, this is all about understanding the overall user
experience.
It's the aesthetics and it's page load time.
You then want to share all that stuff with your VIP's.
Get them to understand what your users are really experiencing.
Get them to understand what it's like on mobile and globally.
Get them to understand also how this stuff impacts the business
metrics, run some experiments and share that data with them.
Once you get a very important person to say,
this is important or just a few find that it's awesome enough
to say this is important, educate your team.
Do those brown bags and lunch and learns.
Make sure that your surfacing that data in their daily work
flows.
Make sure also-- your celebrating things.
Again this is a journey right?
You may be at any step in this process
and you might have to start again gathering more data,
getting more VIP buy in, educating the team again.
New designers and developers will join
and you'll have to do more brown bag lunch and learns.
New technologies will emerge and you'll
want to share that data and that new information with them.
You'll find new wins and you want to celebrate that stuff.
This is a continual journey and continually positive process.
PAUL: OK.
So, as we said at the start we're
in this period of dramatic change
where we're seeing mobile traffic go, whoosh straight up,
and we're facing this world where essentially we
need to deal with these performance constrained devices
like building for this tomorrow that we're looking
at involves facing that fight and saying
you know what most of the devices that people are going
to be using are not desktop, they are not powerful devices.
The thing is it's not really tomorrow it's kind of now.
And that's the thing about dealing with performance
is it really starts and it can and should start today,
but the only way you're going to get to that point
is if absolutely everybody in your organization from the top
to the bottom cares deeply about performance and its impact.
And the only way you're going to get
there is if you start building performance cultures.
Thank you.
We'd love your feedback.
Please do snap the code or take down the URL.
And we'll be outside if you want to ask any questions
and say, hi.
LARA: Come hang out.
PAUL: Yeah.
LARA: Thanks everybody.
PAUL: Thank you.