Hi, everybody.

It's Hanna with FullStory here with another session of FullStory LinkedIn Live.

I'm so excited to be here with Tommy Noonan, a product manager here at FullStory.

Yeah.

Thanks, Hanna.

Happy to be here.

Thanks for joining us this Friday afternoon.

We're coming at you from our FullStory headquarters here in Atlanta, Georgia.

Today we're going to be talking about product metrics.

Just a reminder, this is a live session, so please feel free to leave any comments below.

If you have questions, we'll answer those later on in this session.

Also, we're going to be talking about metrics today, and if this is a topic that you're

really excited about, we're having a webinar actually later on in June with Amplitude.

We'll be talking about north star metrics and how do you create north star metrics,

what does that mean for your business.

There should be a link in the comments if you're interested in joining that webinar.

Please do sign up.

Tommy, just to kick us off, tell us a little bit about yourself.

How did you end up at FullStory?

Yeah, for sure.

I actually started my career at Accenture as a systems integration consultant.

I did a lot of things like building custom solutions for more larger enterprise clients.

Then I actually came to FullStory from business school.

I interned at Google as a product management intern on their mobile search team right before

switching to here.

Cool.

How did you fall into product management?

Yeah.

That is a great question.

I feel like when I got to grad school, I had never heard of a product manager before, but

I knew I really wanted to work with data to make decisions, be in front of customers,

and affect the development or the strategy of the product.

Everybody was like, "Yeah.

It really sounds like a B2B product manager."

I was like, "Yes.

That's what I want to do.

That's what I want to get into."

That's awesome.

You're obviously very interested in data and how we can make better decisions using data.

How do you think about product metrics?

Yeah, for sure.

To me, product metrics are all about decision making.

They should be able to enable your organization to make better decisions faster.

I kind of think about that along two axes.

First is strategically.

It's, "Are you tracking the right metrics?"

I think about that as I want my metrics to tell me are customers being able to, I guess,

complete the use cases that they're coming to our product to do.

Operationally, that's all about, "Are the metrics easy for everyone to interpret, to

read?

Is everybody on the same page?

Do they know where to find them?", but more of like actually once you have the metrics,

what do you do with them?

Yeah, because you have to do something with that data, otherwise it's not operationalized.

I hope people do stuff with it, at least.

Yeah.

What's the difference between a product metric and a KPI?

Yeah.

I actually think of these as fairly similar.

At least, I use the two terms pretty interchangeably.

Yeah.

Do you think that PMs are the best person to be the owner of product metrics?

I actually do think PMs should be the owner, but it needs to be a collaborative process.

What I mean by that is I'm definitely a firm believer that if everybody owns something,

no one owns something.

Kind of a tragedy of the commons type thing.

Like I said, it needs to be collaborative.

There are some things that maybe like an engineer or a designer might understand a little bit

better than a product manager.

Product managers should probably work with their customer success teams and their support

teams to get a better understanding of what their actual customers are saying when they

start to develop metrics that they're going to actually make decisions off of.

That makes sense.

We talked last week about cross-functional collaboration, and it seems like that's just

a recurring theme that I hear when I talk with PMs, both here at FullStory and at other

companies, is no matter your role as a PM, whether you're focused on data or launching

new features or whatever, it's really you're connecting to other people across the business

to understand how do we move forward.

Yeah, and that's a great point.

Product metrics are important because they help create that alignment between all these

different organizations at your company.

Without product metrics, it's hard for everybody to speak on the same page.

It's hard for people to prioritize things if everybody is kind of focused on different

metrics or different ways to think about things.

I feel like the mental resources that we all have are kind of the scarcest resource that

we have at companies like this, and so being able to prioritize things quickly and effectively

is super important.

Yeah.

What makes a good product metric versus a bad one?

Is there a bad one?

I think there are probably certainly bad ones, but it also depends on what you're trying

to track.

Sure.

Churn, to me, is actually an example of potentially a great product metric, like what better way

for someone to tell you that they're not getting use out of their product for them to actually

stop using you and churn?

But it can also be a bad product metric because if I'm looking at churn, it's kind of too

late.

I would like to know that customers or users aren't completing their use cases well before

they actually make the decision to quit.

Yeah, so you need some of those leading indicators.

Exactly.

Yeah, definitely.

We're a quick growing company.

Let's talk a little bit about growth.

How do you think product metrics change throughout growth?

Yeah.

Product metrics definitely can and should change as the company grows, and I definitely

want to reiterate that it all goes back to the customer and how your product is evolving

to meet your customer's needs.

I think an example of that would be like at FullStory, we probably originally found product

market fit more in the SaaS support use cases.

How do we best serve those customers?

They were coming into FullStory every day to analyze different customer trends, watch

sessions of people encountering bugs, etc.

That was something that we said daily active users is very important for us at that time.

As our product has evolved a little bit, and as we start tackling new use cases, such as

conversion rate optimization, product manager use cases, we decided that daily active user

isn't a necessity for these users, so maybe weekly active is the right way to start thinking

about this.

There's definitely never going to be like a light that comes on that says, "It's time

to change your product metrics."

Nothing is black and white, but I think the perpetual, constantly reevaluating who my

customers are, how are they most getting value out of my product, is the best way to really

decide what metrics should I be tracking.

Yeah.

That makes a lot of sense.

For those of you who are joining us, I'm talking with Tommy Noonan.

He's a product manager here at FullStory.

We're talking about metrics.

If you have any thoughts on product metrics, either ones that you think are really great

or other ideas about how you measure engagement with your app, please feel free to post those

in the comments below.

We talked a little bit about how as your company evolves, your customer base usually evolves,

your market fit evolves, and your metrics need to evolve as well.

How do you keep your metrics fresh?

Yeah, for sure.

I definitely recommend ... A, it's kind of like what are the customer interactions.

You need to be very in tune with like what your customers are doing and what are the

most important features and what are the core problems they're trying to solve.

Yeah.

That's certainly key.

We've actually developed a north star metric, which I know we're having our webinar on in

a couple weeks.

It's a way for us to say, "There's a lot of different metrics out there.

There's probably a lot of different use cases people are trying to do, but this is the thing

that we hold, I guess, above the rest of them."

An example of that at FullStory is we actually track weekly active explorers, so users that

come into our app once a week and use search or segment, as kind of like our metric that,

as of right now, we're tracking as the core metric for FullStory product team.

Yeah.

I think it's really clarifying to have a north star metric because then you're able to say,

"Okay.

We have all of this data, we have all these metrics, and this is the one that we really

care about tracking towards, the one we really want to prioritize."

Definitely.

That goes all back into the alignment of different teams, but it also speaks to what happens

when metrics go up or down.

How do we interpret the change that we're seeing?

In my opinion, interpreting and reading that change is probably more important than the

metrics themself.

When you have a north star metric and you actually take the time to socialize it and

weave it throughout the fabric of your product team, it really can be enlightening in terms

of like, "Oh.

A metric that we have is going down or up, but it doesn't change our north star metric.

That might be something that we prioritize.

This is not the most burning thing on my plate right now.

I can look at that at the end of the week.

It's not a drop everything that I'm doing right now."

Yeah, yeah.

That's a good rubric.

When you think about metrics and socializing them, how do you go about doing that?

Yeah, for sure.

Data is certainly the great equalizer, to me.

I think of myself as a data-driven person, so I probably rely on that maybe even more

than I should, but it always is really helpful when you're going to people that you know

have to be bought in on a north star metric, whether it's product marketers, engineering

managers, what have you, to say, "There's a reason why I'm led to this specific metric."

Maybe it's something that reflects what you think of as like the core strength of your

product.

Maybe it's something that you think of as the users that use this metric tell us that

they're getting the most value out of our product.

Being able to back up why you're choosing one thing or another is, to me, extremely

important when it comes to actually convincing everybody to come around it.

Yeah, definitely.

You talked a little bit about responding when your metric either goes down or up.

How do you know when you should respond versus, "Oh, this is just normal fluctuations based

on things that are happening"?

Yeah.

This is definitely like a million dollar question, and I'm sure it keeps a lot of people up,

myself included, at night.

One of the ways I tackle that is with the north star metric.

It at least gives me a little bit of clarity in terms of, "Okay.

This is going down.

It's important, but it's not necessarily the most burning thing on my plate."

Also, we have the idea of like KPI or metrics trees.

Those are probably best represented by how are our metrics related to each other, or

how does one thing affect another.

Yeah.

Taking our weekly active explorer, for instance, I think of it as a weekly active explorer

is a combination of how many monthly active users we have, what percentage of those monthly

active users are coming into our app on a weekly basis, and then lastly, what percentage

of those weekly active users are actually exploring.

That way, when I see my north star metric go up and down, I can kind of immediately

pinpoint, "Oh, monthly active users is down.

It's Christmastime, but the people who are actually here, they're searching, they're

coming into the app frequently."

That's something where I say, "Okay.

No worries."

It just gives you a head start and a little bit of clarity in terms of what to look for.

Yeah.

It seems like you still have to contextualize the data elsewhere though, understanding what

other factors might be at play.

Yeah, absolutely.

Kind of thinking about this in the positive light of what happens when your north star

metric goes up, this is something that we look at really closely when it comes to things

like feature launches or big, drastic changes in a product.

An example for this for FullStory is a few months ago, we launched a feature called Custom

Events.

We noticed that our customers that were using the feature were actually performing better

on the north star metric than our customers who weren't.

That in and of itself was like, "Great.

We released something that we wanted to change our north star metric, and it did."

What was really enlightening for us was actually looking at the KPI tree and figuring out why

the weekly active explorer numbers were going up.

That was when we found super quickly, because we already have, I guess, like a framework

for analyzing these things, "Oh, these were actually never before seen users that were

coming into our customers' apps and using FullStory now because they wanted to use this

new functionality."

That was something that was like, I guess, a great product metric win for us, being able

to predict what would happen and then actually see it happen in our metrics.

Yeah.

I love that.

If anyone who's viewing has any product metric wins you'd like to share, you can do that

below in the comments.

I'm curious.

If someone is watching this and they're wanting to get started maybe creating a north star

metric or becoming more data-driven, what advice do you have for them?

Yeah.

That's a great question.

Let me think about where to begin.

For me, I think it's ... When I think about when I first came to FullStory, it was like

making sure that we had the data needed to analyze the things I wanted to analyze or

start to like string all these things together, eventually bubbling up into a north star metric.

A lot of times at early stage companies, you might not necessarily have an entire BI or

an analytics team that's sole focus is to work on an analytics stack that people can

query and draw inferences from.

Mm-hmm (affirmative).

That was something that was very big for me.

I would say it's understanding what data you have and then how to get the most value out

of it because even though getting the data's important, I think you'll find pretty quickly

that there's a plethora of things you can analyze.

It's really about asking the right questions and figuring out what you want to track.

I think that all goes back to the customer, trying to understand who are the people that

you're trying to serve and what do I need to know to best serve them.

Yeah.

That makes a lot of sense.

You sort of start with that perspective of understanding your customer and then using

that to say, "Okay.

What are the questions that I care about?

And then how can I answer those questions with data?"

Yes, exactly.

Okay.

Cool.

That makes a lot of sense.

You mentioned tech stack.

What is in your tech stack that you love for understanding your users and metrics?

Yeah, for sure.

I don't think this is going to surprise anyone, but I am a very big FullStory user, both the

actual UI and I'm a heavy power user of our data export feature, which is kind of like

the raw data underlying everything that you might analyze in the app.

I also have a SQL background from Accenture, so I love using BigQuery.

That's kind of the, I guess, analytics warehouse that we use to take things from not just product

engagement data, but things like Salesforce, customer support tickets, customer segmentation

data, kind of everything I can meld together in that type of environment.

Then we also have Looker that sits on top of that.

It's been super helpful, honestly not as much from the data visualization side, but more

of just like how is our business logic to convert all of this disparate data into actual

insights.

Looker kind of houses that for us.

Yeah.

That's awesome.

I think from my perspective, I know that I've gotten to work with you on some data projects

and saying, "When are we launching features?

How are people engaging with this?

What's happening?", but it's been really exciting to see you roll out north star metrics here

at FullStory.

I think it's created a lot of clarity around what are people doing on our app and what

do we want to truly be tracking towards.

I think as an observer of that, I can recognize the value of using data to make decisions.

Yeah.

That's definitely great to hear.

The north star metric has definitely been super enlightening for me.

I almost saw it as like product management tech debt.

As opposed to being kind of reactionary every single time we heard of something or something

was going on, we now are at least like monitoring what we feel is the right thing and responding

appropriately.

Yeah, definitely.

That's awesome.

We're going to turn to the audience now.

If you have any questions, please feel free to drop them in the comment section.

We did have a comment.

I think this is great.

It said, "The age old question.

No one grew up wanting to be a product manager", which that certainly seemed to be your experience.

Yeah.

I would definitely say so.

Do you have any advice for people who maybe are considering a career in product management?

Yeah, for sure.

I definitely took one route to product management.

I went to consulting and then business school and transferred in here, but I almost think

that one of the better ways to do it is just to go ahead and actually get in the companies

that have product management positions, even if you're not going to be a product manager.

Just show them that you are, I guess, the right person for it.

There's definitely no right way to get into the field.

Yeah, definitely.

We talked a little bit about someone who's just starting off measuring engagement or

product metrics.

What metrics would you specifically suggest a small team start off with?

Yeah.

That's a great question.

Is it daily active users?

Monthly active users?

It just depends maybe?

Yeah.

I definitely recommend ... Those are probably the core three I would start with, monthly,

weekly, and daily.

Then from there it's like knowing what type of business you are.

Are you transactional-based, that maybe people just come to create a transaction and aren't

really coming on a daily basis to complete tasks?

That might have something different.

You might be tracking transactions instead of just users that are coming.

Yeah, definitely.

What do you think the challenge is for people who are trying to create product metrics?

Are there specific challenges you faced that others might be facing?

Getting everybody to align on metrics is tough.

You're not going to get alignment unless you prove that you're willing to listen to everyone

who has something to say about it.

Listening to people from different perspectives, I think, is super important.

I think data is helpful, and it can help build conviction in yourself before you want to

present an idea out there that you know is probably going to get poked and prodded from

every single way, but listening to people, building that trust, and showing them your

thought process for building up to a north star metric was super helpful for me.

Yeah, definitely.

That makes a lot of sense.

Great.

Thank you so much for being here with us today.

I really appreciate your time and insights.

Yeah, absolutely.

This was fun.

Yeah.

Awesome.

Thank you for watching.

If you've been interested in product metrics, you want to learn more, we're having a webinar

with Amplitude at the end of June, so please join us for that.

You'll be talking about north star metrics.

Yep.

And we'll be talking with a PM from Amplitude about north star metrics too.

If you enjoyed this content, please follow our LinkedIn page.

We'll be back here next week with another LinkedIn Live.

Thanks everybody.