>> Welcome back to Armchair Architects
as part of the Azure Enablement Show.
We've been talking a lot about data this season,
this time the question that comes up is
how do you get any meaning from your data?
Let's go talk to the architects about that.
>> [MUSIC]
>> Hey, well, welcome back to
another episode of Armchair Architects,
as part of the Azure enablement show.
Now we're moving into a topic that we have heard
loud and clear people want us to talk about, which is data.
I wanted to start off at a data 101 level and ask
a very basic question and take it at whatever level you want,
but I would really like to understand the question.
I am swimming and you can't see it,
but there's some over there and some over
there and there's some around here on my floor.
I don't know how to make meaning of this stuff.
This is actually the canonical problem.
Iíve got so much data,
I don't know how to make meaning out of it.
Can we go in that direction?
How do you think about this question of making
meaning from your data?
How do you pronounce it? Is the question.
>> I think this is actually near dear to my heart because this is
just something I'm doing right now for multiple teams here.
But this is also probably related
to our session on dark data which is,
I know I'm capturing data but how do I find it,
use it, and actually synthesize it to answer my questions.
I think this talk is a little bit different than that one
because let's assume that you don't have a data swamp anymore.
Let's assume that you have
a relatively streamlined way of
extracting data from hundreds of thousands of
transactions or mysterious funny messages coming
from little devices that you connected into a Cloud provider.
What is usually lost is context,
which is how do I actually
understand this time series message that I got from
this funny little device which is only identified by
a GUID in the IoT scenario or in the IoT space?
How do I know where this thing was,
who manufactured it,
when it's last firmware update was,
what it's making, and then
where it actually sits in physical space.
All of those things are inherently absent from the little message
that it sends into the Cloud which
eventually finds its way into your data lake.
In order to answer questions
which many of our customers want to have answered,
which is, what's happening right now?
How efficient am I?
What's going to happen and what should I do about it?
Contexts associated with data,
which is commonly in my estimation referred to as semantics,
which is the meaning that that data holds
and the context associated with it is super important.
>> Uli, it looks like you're about to say something.
>> Again, meaning and
contextualization is obviously very important.
But then think about the other piece that's often the case.
If you look at data models oftentimes
the data designers don't really specify what they meant.
Is this pounds in pressure,
pressure in pounds,
or is this in kilowatts,
or whatever the measurement unit for
example is often omitted because people know,
hey, that's how we count.
That's really another element of
this contextualization that often
gets overlooked because people "know".
I'm sure you remember
the famous NASA event
where one guy thought they were working in metric,
the other one thought in imperial,
not a good thing that lead to
a loss of a fairly expensive satellite,
a rocket or whatever it was.
I think it's really important to think
about all the aspects that Eric talked about,
but then also bring something very simple.
Hey, what is the unit of measurements we are using
Here, as one example?
>> That's an example of definition and being important to
context if I'm understanding you right in explanation.
I get the sense from our previous conversations that in
addition to that context also includes relationships.
Because it's really cool that you get that one thing going, boop,
boop every whatever boop length of time is.
But that's if you're talking about a chemical factory.
The thing that's going boop,
boop next to the thing that's going boop,
boop, boop, boop, boop, boop.
I don't really know what they're doing.
I don't really do a lot of work in IoT.
I'm assuming that's the noise they make.
Also matters. There is a relationship between those,
Ahead of you behind you,
in front of you. That's our stuff.
>> Yeah.
>> How do we think about relationships
when it comes to meeting and data and contacts.
How do we describe them? How do we think about them?
>> I think about it in three ways.
There's entities,
attributes, and relationships between the entities.
Those are things in which from an architecture perspective,
from a data modeling perspective, that's your foundation.
There's got to be a way,
a technology or a model in which you can express those things.
The way that you do that is you use languages like RDF or
Label Property Graph or any of
these modeling languages to say this is what my world looks like.
These are the things that I care about.
Some of these things actually correlate to physical objects,
like in a traditional digital twins use case.
Some of them might be business objects like
work orders or supply contracts or things of that nature.
All of them exist as
entities which have attributes, which are properties.
Those properties have meaning,
like Uli's example of units of measurement.
Then there's relationships between them.
That's the way in which we want to
present as architects and solution builders,
we want to present that to our customers and our end users.
Not necessarily this table has an associative entity which
were links to this table and use
the primary keys and foreign keys associated with it.
I don't know the temperature measurement,
metric or unit of measure.
We want to be able to say, hey,
here's an experience that says
these are the entities and attributes and
relationships that you as
business users or regular users understand.
If you're not sure what something is,
you can mouse over it and gets semantic meaning.
This is the unit of measure for this temperature.
That actually brings together contexts and contextualization.
But in order to get there,
you have to be able to tie these sets of data together.
You have to, as Uli says,
conduct data fusion between
multiple data sources in order to achieve this.
>> Well, this is a question I have for Uli.
I would guess that all relationships are not
always known prior ahead of time.
To get meaning out of your data,
sometimes you're constructing relationships
after the fact or in process.
How do you think about that.
There's the things you already knew that was connected.
But how do we think about
the things that we didn't know or that we're
discovering or working over our data
to find how did those relationships work?
>> Well again, there are two models for how you think about this.
First of all, it's a temporal alignment.
Where the time measure is the key relationship edge.
To use the Graph language that Eric was always so fond of.
Graphs are one way of describing things that are otherwise.
Describe things. Iíll give you a very concrete example.
I'm sure all of you have seen the Boston Dynamics Spot robot,
a little doggie-like thing.
That's thing has the ability to host
multiple sensor packs but only one at a time.
You effectively, let's say, or three sensor pecks.
One for vision, one for audio, whatever.
You put one on there.
It runs around, does its thing.
Then an hour later it picks up another sensor pack.
How do you model that and how do you make sure that
you understand that hour one,
I use the vision pack in hour two
use the whatever motion pack or whatever,
and that you can actually reason over that.
Because the dog, the robot is
always the same but the sensor pack, isnít always the same.
But the combination is actually the interesting one.
This is not a discovery one.
This is something that you schedule and you are set up.
You have to now reflected in your system.
Your system has to be able to say, Oh,
I want to know what the robot actually
was carrying at hour X of day,
Y and so forth.
Because your data analytics will be skewed differently.
If you're don't really understand that.
Obviously then there's the pattern
of the way the dog robot is walking.
You need to understand locality,
location, all those kind of things.
Then other things that come from outside data.
Humidity plays a very big role in
manufacturing for example especially
when you do process roll stuff.
It really impacts what's going on
or the weather in terms of ìis it humid?î,
ìis it hot?î, ìis it dry?î, ìis it cold?î
Those things are really important.
That's when you have to
understand what is the data that you care about,
what is the influencer?
Then most likely you have to expand your view beyond
just the raw data that you have to get to
contextuality that actually really makes sense.
That's part of the discovery,
is what really influences your dataset.
You have a dataset that says the temperature inside.
I don't know what thing is this?
Well, it doesn't really matter.
Is this now impacted by the surroundings because it's
now minus whatever 15 degrees Celsius out there.
Does that have an impact on how much you have to
heat the pipes in order to keep the oil flowing or whatever.
All of these things are super important.
That's the quote, unquote discovery from my perspective.
Then you have to model it so that you
can persist it and then bring it back.
When somebody asked the question,
what happened to this pipe?
or this oil in that pipe.
At that time, you know exactly
what the elements of data that you need to bring back.
That's unfortunately experience and experimentation.
Are the two pieces that I would put there.
>> I know that Eric's about this case.
I just want to make sure I get both
of your things because you said there were
two things associated with the question about relationships.
One was a temporal proximity,
and what was the other one?
>> It's effectively, let's call it causality.
Where you have causal data like temperature on the outside,
humidity and those things that you can't influence.
They are very zen like that, just are.
But the impact and influence what you measure
and what the meaning of that measurement
actually is or the impact of that is.
>> Cool. I just wanted to get that settled.
Eric, what were you going to say about this.
>> Actually Uli's example,
maybe think of something and for many of
our viewers that had been in the data warehousing space,
this is not a new problem.
For those of my folks that are,
data warehouse tool kit fans Bill Inmon or Ralph Kimball,
depending on which side of the world you're on.
We used to do this in the data warehousing days.
We took a fact table which is the raw data,
sales data, let's say.
Then we had dimensions around it.
We had a salesperson dimension,
we had a time dimension and a product dimension.
We could figure out who sold what and when and in what quantity.
Now, one of the challenges associated with
this Uliís example made me think of is David,
let's say that you and I are working for
the same company and I inherit your sales territory.
Now we have a history problem in
which I can't actually historically take
credit for all of your historical sales if we
make that change in the salesperson dimension,
we have what we used to call it and probably still
do call a slowly changing type two problem.
What that means is as much as you
define the entities, relationships,
and attributes that will help you
contextualize raw transactional data.
You actually have to make sure that you focus
on how those models themselves will change over time.
That you're not actually looking back in
history with today's understanding of how those entities,
attributes, and relationships are implemented.
It's a very old problem that's going to only
be exacerbated by the different speeds,
feeds, and data sources that we have in today's modern world.
>> I have to admit, I don't know what that sound is.
But what that sound does remind me of
is that I think what we're going to do is be heading.
I mean, it seems like
the direction we're heading with this conversation which is
a place I was hoping where to go is a little bit
now towards the implementation side of things.
If it's okay with you,
let's cut off here and then come back and dig into some of
the implementation size of this data stuff
that we've been talking about. That sound cool?
>> Sure.
>> Great and help whoever it is,
who's sneezing in the background?
>> It's not me.
>> Shawn, are you going to live?
>> Thank you, folks. Eric and Uli and Shawn for joining us
today on this episode
of Armchair Architects as part of the Azure Enablement Show.
>> [MUSIC]