>> Hi, I'm Lachie.
>> Hi, I'm Christine.
>>And I'm Aneel.
>> This is the Open Source Show.
Welcome back, I'm Lachie Evenson,
a PM on the Cloud Native Compute Team at Azure.
>> Hi, I'm Christine Yen,
one of the co-founders of Honeycomb.
>> I'm Aneel Lakhani. I run strategy at Honeycomb.
>> Today, we are going to be talking
everything observability.
So, we have two experts here today.
Can you tell us a little bit about what observability is?
>> Basically observability is being able to ask questions
about what your software is doing and
get answers about what it's doing.
So, different from metrics or logging or monitoring,
it's not just knowing
whether or not something is up or down,
it's being able to say, what is it doing right now?
Oh, it's doing this, over here, for that person.
>> In the last few years observability has
become really in the forefront of everything,
why do you think this is?
>> Everything is getting more complicated these days.
Back in 2005, you could have
one LAMP server doing a couple of things.
If something went wrong, you go look,
you go read the the logs, traffic was lower,
you could read the logs with your eyeballs.
Today, you've got instead
of one or five or 10 app servers,
you've got hundreds of containers,
thousands of Kubernetes pods.
Everything running in all these different ways
to tease apart what might be going wrong,
what might be misbehaving
and really being able to do what Aneel said,
answer these questions about
your systems using the data that they output.
>> It's for anyone who
both builds and runs their software,
so this could be a software engineer,
someone called a developer,
someone called the DevOps engineer or a site reliability,
like the title doesn't really matter.
What's important is, are you in some way responsible for
this code running somewhere
and does it do something for somebody?
Do you need to know what that is? You could be
a product person, you could be in support.
>> Some of our customers have
their product managers
very carefully watching performance.
So, they use something like Honeycomb to tease apart.
Okay, well I'm watching this one performance metric,
but let me filter out this endpoint that
I know is slow and let me look at
this endpoint today and then on this client platform.
They're able to pinpoint just what they're looking for
that day and look for something entirely
different the next day and they don't
have to predict ahead of time,
what metrics they might find interesting,
they don't have to build these static dashboards.
>> They don't have to go request access to data.
They don't have to go ask someone to
create a metric for them or to write a query for them.
The idea is that anyone who is
responsible for software should be
able to both create the data they need to
understand what's going on and then ask the questions.
>> You introduce this term
of observability-driven development,
can you tell me a little bit about what that is?
>> Fundamentally, it's this idea
that if you're going to write software
or create something,
you have some hypothesis about what it should be doing.
So, can you answer that question?
Is it doing the thing I wanted it to do,
for the person I wanted to do it for,
in the way it's supposed to do it, or isn't it?
Then can I take that answer and
have it influence the way I do development?
Because things like testing shouldn't
end at your machine,
or your testing environment,
or your staging environment,
or your CI/CD system,
they should continue all the way through
the actual usage of the software.
>> Observability, it's all just about
answering questions about your systems using data.
One of the things I love about
tying it to the development process,
this phrase is that,
people are really forced to think about, okay,
when I'm building something
and I've pushed it out but it's behind a feature flag,
how can I use information about how it's behaving in
production to decide whether to
flip that flag, whether to make more changes.
When you're writing software, there's
all sorts of points that you can and should be using
information about what it's doing in production to
inform your decision of what edge cases really matter.
'Hey, is this edge case maybe not an edge case?'
There's all sorts of questions that
a lot of product managers tend to
hypothesize about where in fact,
the engineers or anyone can go find out what's normal?
What's my normal latency?
How many customers are actually experiencing this?
By making it more comfortable and more native to feed
information about what's happening in
production into the development process,
it helps us write better code,
ship more stable software,
deploy faster, and deliver
better experiences to their customers.
>> This makes the point about it's
not just for engineers or operators.
A product manager needs to know is the thing I
pushed or promoted or launched
actually being used in the way I expect it to be used,
and if it's not, how can I change that,
or how should I make
different decisions about the features that I make.
This is an ideal outcome for us.
You think about empowering one individual developer or
person responsible for software
and that should expand to their entire organization.
If we have both free access
to produce data but also access
that data and to ask questions but
also share the answers around,
then a person from one team should be able to
find something to solve for another team.
>> One of our secret missions at Honeycomb
is to democratize this power.
Too often we have these engineering teams
where there's one single repository of expertise
and we want that to not be the case anymore.
We want people to be able go on vacation
and people to be able to look things up themselves.
>> So, there are tons of tech conferences.
The reason we put on o11ycon is
we are trying to build a community of practice,
not of a specific role,
or not around a specific technology,
or a specific piece of software.
So, we're trying to get together
people who want to be able to
ask and answer questions
in this way that we're talking about.
>> Because observability is such a new concept,
it's not really a venue where you can go pick
the observability experts and put them
on stage and have him talk for a full day.
Instead, we structure the day around a number of
open space discussions and you just have
these clusters of folks
form around a particular topic
they're interested in and you'd
have the junior engineer from
a small engineering company on
the East Coast talking to
an engineering director in San Francisco,
all about how they use data,
how they spread a culture of answering
these questions to drive their development
and swapping techniques and it's very cool,
made you feel like you're at the beginning of a movement.
>> I wish I was there. We have
some scenes to share on video.
>> I'm really trying to hammer down the point that
context really turns unknown unknowns,
where you don't know what you don't know,
into stuff where you're like,
I have an idea of what could go wrong.
>> We have moved from understanding
a system in some formal way
to running a system and
doing stuff and seeing what happens.
>> You better believe that the developers are held
accountable for making sure that
their applications are operable.
>> I think if most of us stop and think,
''Is our instrumentation good enough
to find these opportunities,
everywhere we have them in our app?''
Most of us if we're being
honest with ourselves would say no.
>> That looks fantastic.
>> The whole point is that if we can get
a lot of people all over our space and
in different regions talking about how
to ask and answer questions
in a way that's free for everyone,
so that we can all get together and break down
some of these artificial silos we've made
between teams and between how we think about
getting data and answering questions.
>> But how is the best way for me to get connected?
If I'm really into observability,
what steps can I take to get started?
>> A great way to start is
to look at the observability guide,
we have on our website honeycomb.io.
From there, you should find a lot of other resources.
>> I look forward to being a part of
that community and watching it grow.
Thanks for joining today. This is the Open Source Show.
You can go for more information to
opensource.microsoft.com and the links
will be in the description. Thanks for joining.