[Music]
hello welcome to Microsoft Mechanics
live coming up we're joined by power bi
expert just see analytic to take a look
at power bi is roll as part of the power
platform and also helping you to prepare
your data across sources now we're going
to demonstrate the latest power bi
capabilities to help you discover and
explore insights in your data including
automated machine learning for
predictive modeling new AI
visualizations with route decomposition
trees for root cause analysis and easier
ways to integrate your data using new QA
visualization please join me in
welcoming though Justina Lucia Nick from
the power bi team welcome oh it's
fantastic to be back thank you awesome
so I think a lot of people are probably
aware of power bi role is really
building custom reports and all the
awesome data visualizations that you can
do with it but a pivotal yet not very
well known role that it plays is also
helping you to prep and consolidate your
data across your apps and your systems
and all your services and all the
databases that you use every day right
so most people know power bi is this
powerful data visualization tool it lets
you build beautiful reports unless you
find insights inside your data but it's
part of the power platform which
comprises of power automate for
automating processes power apps for
taking action part bi lets you do more
than just analyze your data it actually
plays a really important role in the
data consolidation process super bi lets
you pull data from lots of different
sources cleanse it bring it together
insert it in the common data model
format or CDM format this means that our
bi can make use of this data but so can
other services like Azure data BRICS
Dynamics 365 Adobe and many more and
just a pause here for a second for those
people that are new to the CDM or common
data model what's its function yes this
is an open format that power bi uses as
you prepare your data power bi states
this data into the azure data Lake in
CDM folders CDM folders are made up of
CSV files and Jason descriptor file is
called modeled adjacent files
and these files can be read and written
by the Power Platform by Azure services
by Dynamix 365 and others you can think
of it as like a common language that
lets us all collaborate on top of the
same data right so now your data
scientists and engineers can actually
work on top of all your organization's
data and power bi really plays that
important role in this case to do things
like service integration and also the
data level but how does this all work
yep
so data preparation on top of the data
Lake and power bi is done using
something called data flows so let's
take a look here I'm preparing some
supply chain data and you can see over
here on the left hand side I'm bringing
in data from lots of different data
sources now all of these purple boxes
that you see over here are called data
flows so these data flows actually
represent a lot of data transformations
that are happening behind the scenes
okay now these different data flows can
also be chained together to consolidate
all of my different data and as let's
say I'm to refresh one of these data
flows over here power bi takes care of
all the functional dependencies so I
don't have to worry about things like my
data being overwritten now behind the
scenes all of this data is actually
being stored inside the azure data Lake
so if I pop this into a ListView and I
open up my Azure storage Explorer
side-by-side you can see over here that
I'm connecting to my Azure data Lake I'm
inside my power bi blob container and
I'm inside my supply chain view
workspace now all of these folders that
you see over here are CDM folders and
they're just representing all the
different data flows I have inside power
bi right so if I go ahead and jump into
one of these you can see all of these
different folders are actually storing
CSV files and most importantly I have my
model dot JSON descriptor file over here
okay that power bi can read and write
and this allows my power bi analyst to
collaborate with others inside the
organization so that means if my data
scientist for example is working inside
an azure notebook using Python code they
can go ahead and read and write the same
data flows as well right and it's really
powerful so you basically end up kind of
with reusable ETL blocks and you've made
up the data readable and you've made it
really better from across apps and the
analytical services that might be able
to tie into it
so now you've got your data sources all
in why don't we move on to some of the
new things that we can do with it yeah
so we're actually adding a lot of very
powerful enrichments that you can do
with data flows and power bi and weather
them which I'm very excited about is
actually the ability to build machine
learning models directly inside car bi
so let's take a look at how we could do
that I'm gonna jump into my sales order
data flow and never here I have this
backorders entity so in the supply chain
world if something is on backorder it
means it's not available for me to
purchase it's out of stock so that's a
bad thing we want to minimize backward
right so what I'm gonna do is I'm gonna
click on this little icon over here and
I'm gonna add a machine learning model
okay so the first thing I need to do is
I need to choose a field I want to
predict and I have this true or false
field that tells me whether a product is
on back order or it's not on back writer
right now I'm gonna hit next and here
power bi is actually gonna take a look
at my data and it's gonna recommend what
sort of model we should be building so
over here it's recommended me a binary
prediction model because some things are
in backorder or not in backorder exactly
so here I can choose a state to focus on
and I can change the labels that should
say high risk or low risk yep and then
the last step of what I have to do is
choose what kind of data I want this
particular model to learn from so over
here probably died as actually does an
automatic selection for me but if I want
to tweak this around I of course can do
that too and that's it in terms of setup
now when I train this model a couple of
things are actually going to happen
behind the scenes automated ml is going
to look through all the columns I
provided with and it's gonna do feature
selection it's then gonna iterate
through lots and lots of different types
of algorithms to try and figure out
which are the best algorithms it can
learn from for my problem and then for
each of those algorithms is actually
gonna tune those algorithms doing hyper
parameter tuning so it's gonna keep
iterating through all of those different
steps to build me the best possible
model that it can right so it's actually
creating lots of different models behind
the scenes and automatically then trying
to find the best and most optimized
model in your case yeah exactly this was
all done automatically we're really
trying to democratize machine learning
for a business analysts so once this
model is built I can just use this as
part of my data flows pipelines and
score new fresh data coming in okay and
then let's take a
the service reports you can build with
this information let's do it so over
here you can see a report where I've
actually added a backorder risk score to
all of my data and so I've got this
particular product which is a high risk
of being on backorder and ashlee
automated ml doesn't just produce these
risk scores it actually produces row
level explanations for me as well so
over here you can actually see that this
particular product is at high risk of
being on backorder because inventory is
low we're not doing a great job
forecasting it and we're behind on
manufacturing okay so now with automated
ml that business decision-makers are
also not just presented with
probabilities but are easily even able
to dig in and kind of understand why
something might be at risk so it also
makes the insights themselves a lot more
actionable to be able to take action
yeah that's right you know and users can
be a lot more proactive with addressing
future problems rather than just
reacting to them however we know not
every single user is going to be
building machine learning models in
power bi and we really want to
democratize these sorts of insights and
make them available to everyone so one
thing we're doing is we're building AI
visualizations for interactive
exploration on top of your data ok so
I'm going to jump into this other tab
we're gonna stick with our backwards
problem and here we're gonna use the
decomposition tree to analyze this data
so what I'm going to do and what's
really cool about this visualization is
I can drill into any column I want in
any order right this is the little tree
that's gonna keep this is a baby tree
yes so it's gonna keep growing so maybe
we want to break it out by let's say
forecast bias and the tree automatically
starts expanding right maybe we want to
focus on our accurate forecast and jump
into demand type and the tree just keeps
growing but now where the magic comes in
is maybe I want to understand where all
of my backwords are coming from and I
don't know what to drill into next it
could be a lot of factors it could be
white bulbs might help you out yes
that's a great point Jeremy yes so over
here I can just ask power bi to find any
high values or low values in my data so
I can select this the tree grows and
tells me I should be looking at my
patient monitoring product type and if I
do this again I can see I should be
looking at a particular plant and this
is a really useful way for doing root
cause analysis and it's fully
interactive with the rest of my report
as well awesome so now what if I don't
want to determine the fields to bring in
but just want to maybe ask questions
kind of using human speak to the data
huh
so we actually have just introduced this
new Q&A visualization to power bi Jeremy
okay
so I'm gonna jump into this particular
page over here we've already started
plotting this new cue a visual and you
can automatically see that we have a
bunch of suggested questions that just
come up and these are just automatically
generated by power bi by analyzing my
data model so when I select one of these
we just go ahead and plot a
visualization as the answer right so you
me have to know the questions to ask and
people will remember a Q&A more probably
as a pop-up feature at the kind of the
highest level of power bi but now you
can use this visualization capability as
part of the tiles and QA is really part
of the sub reports themselves yeah
exactly so now this just becomes another
visualization in my report and as I go
ahead and interact with it the filters
flow in and out of the visual there's a
lot of other cool features that I wanted
to highlight let's imagine I want to ask
about something like back order a dollar
amount by area now power bi is going to
answer this question but what's
interesting over here is you can see it
is actually highlighted area in red that
means it didn't understand it it was
able to answer the question anyway and
the reason it was able to do this is
because we're integrating with the
office of the Souris so we're
automatically annotating the whole data
model with synonyms yep so region is
defined in my data model and we just
figured out areas a synonym for it okay
so what happens if you've got some
bespoke terms maybe that are for your
business only that aren't part of normal
dictionaries that's a good question so
let's look for something like maybe
percentage out of stock by product type
so now what's gonna happen is you will
see QA was not able to understand this
question we've also completely revamped
the Q&A tooling in power bi so I'm gonna
jump into the tooling over here and I
can just submit this question and what
we're gonna do over here is I'm gonna
select the whole term I want to define
percentage out of stock and we're gonna
use QA just to Train QA so we can say
percentage out of stock refers to
something like percentage on backorder
and you can actually see the little
preview is going to pop up and show me
now that this is getting the correct
answer for me cool so now as an analyst
is there anything that you can do to
help figure out the types of questions
that people are often asking against the
different data sets that you have here
so
kind of know where to focus on what
reports to build and kind of where to
Train it yeah so I don't have to only
rely on let's see my own intuition to
figure out what vocabulary I should be
adding to QA and the reason for that is
we have this new review questions tab
over here where I can actually see that
all of the questions that are being
asked in QA in the power bi service so
these are my end users interacting with
Q&A in the service and I can see this
questions that they're asking I can even
see if they've requested a fix for this
particular problem as well so now I can
go ahead and add this new vocabulary to
QA nice so a really great overview of
how you can actually use power bi to
prep your data apply machine learning
and also great new visualizations all
these great Q&A updates even to train
the systems but where can folks go to
learn more about these and other updates
yeah
such try out our new decomposition tree
visualization and the Q&A capabilities
as of the November power bi update
you'll be able to download the new power
bi desktop at power bi comm for data
flows and also now you can try out these
capabilities in the power bi service
today really great recommendations of
course and keep watching Microsoft
Mechanics for all the updates across
power bi and more thanks for watching
we'll see you next time goodbye for now
[Music]
[Applause]
[Music]