OKAY. HELLO EVERYONE. I'M CHRISTIAN
WADE. THANK YOU FOR JOINING ME THIS
MORNING. SO WE'RE GOING TO BE TALKING
ABOUT THE ENTERPRISE DATA SETS IN
POWERBI. JUST TO GIVE YOU A BRIEF
HISTORY WE'VE TRADITIONALLY PROVIDED
SERVICE FOR ENTERPRISE SCALE DATA
SETS WITH MANAGEABILITY AND OPEN
PLATFORM CONNECTIVITY. WE WERE --
I THINK IT'S FAIR TO SAY THE ONLY
TRADITIONAL BI VENDOR THAT EMBRACED
SE SELF-SERVICE BI. THAT'S WHY WE'VE
BEEN RECOGNIZED BY THE RESEARCH
AGENCIES. WE TRADITIONAL HAD THE
SERVICES. ONE FOR SELF-SERVICE,
AND ONE FOR ENTERPRISE BI. WE AS
WELL AS THE INDUSTRY IN GENERAL
IT MAKES SENSE TO HAVE A CONVERGED
ALL INCLUSIVE PLATFORM THAT WORKS
FOR BOTH WHICH FOR US WILL BE POWERBI.
WE HAVE TAKEN ALL OF THE EXPERTISE
AND FUNCTIONALITY WE HAVE IN OUR
SERVICES WE BUILT UP OVER THE LAST
20 YEARS, AND WE'RE EXPOSING IT
IN POWERBI. POWERBI AND SHARES THE
SAME ENGINE. WHICH MAKES IT QUICKER
FOR US. SO WE'RE GOING TO TALK [INAUDIBLE].
AND SO WE RECENTLY LAUNCHED X M.
L. A. END POINTS. WHAT THIS MEANS
IS WE'RE AND CONNECT TO ANALYSIS
SERVICES SERVER, SO YOU'D HAVE POWERBI
AND EXCEL AND THIRD-PARTY DATA VISUALIZATION
TOOLS THAT WOULD CONNECT TO OUTSIDE
SERVICES FOR WHATEVER REASON THOSE
CLIENTS WANTED TO DO . SQL PROFILER.
EVERYTHING TO CONNECT TO THE POWERBI
SERVICES. WE CAN JUST EXPOSE THE
ANALYSIS SERVICES ENGINE IN POWERBI
AND ALL THESE TOOLS CAN CONNECT
STRAIGHT TO THE POWERBI WORKSPACE
AND YOU GET ALL F T OF THE SERVICES
WITH THE M L MLA SERVICES. YOU WILL
CONNECT THROUGH POWERBI WORKSPACE,
AND SO THESE ARE ALL TRADITIONAL
ANALYSIS SERVICES CLIENT TOOLS.
SQL SERVER. THIRD-PARTY TOOLS. THAT
STUDIO. EVERYTHING JUST CONNECTS
TO THE POWERBI WORKSPACE URL. AS
THOUGH IT WERE AN SE SELF-SERVICE
SERVER. I'M GOING TO SHOW YOU A
DEMO AND INCREMENTAL REFRESH. WHERE
A LITTLE PBIX THAT'S A FEW MOOUING
BYTES BIG. AND YOU UPLOAD IT TO
THE SERVICE AND THE SERVICE IS EASY.
IF YOU'VE GOT MASSIVE DATA SETS.
BILLIONS OF ROWS. BILLIONS OF TERABYTES
IN YOUR WAREHOUSE. WE TAKE THAT
DATA FROM THE DATA WAREHOUSE, AND
WE COMPRESS IT INTO MEMORY, AND
WE LOAD IT INTO MEMORY WITH THESE
IMPORT MODELS AND GIVE CUSTOMERS
BLAZING FAST PERFORMANCE. IT DOESN'T
MAKE SENSE TO RELOAD THE DATA EVERY
SINGLE TIME. IT WOULD PROBABLY TAKE
FIVE DAYS TO DO THAT. SO YOU'RE
GOING TO DO LIKE AN INCREMENTAL
DATA LOAD. YOU'RE GOING TO PICK
THE DATA THAT'S CHANGED AND ONLY
REFRESH THAT DATA THAT'S CHANGED.
AND YOU'LL REFRESH YOUR SLAs. YOU
WANT TO REFRESH THE DATA THAT'S
CHANGED. IT'S MUCH QUICKER. SO YOU
DEFINE THIS POLICY IN DESKTOP. SO
I'LL GO AHEAD AND SHOW YOU HOW TO
DO THAT. SO I'VE GOT A DATA SET
HERE. LET'S PRETEND. I'M NOT SURE
WHY -- OKAY. IT'S NOT SHOWING MY
SCREEN. SORRY. LET ME TRY THIS.
SO I'VE GOT THIS DATA SET. THIS
PRETEND THIS TRIP DATA SET HAS 10
BILLION ROWS. I WANT TO DO THE MODELING
IN THE POWERBI DESKTOP. I DON'T
HAVE THE RESOURCES ON MY MACHINE
TO DEAL WITH THAT ANYWAY. THEN I
WANT TO REFRESH. FIRST TIME I DO
A REFRESH. SUBSEQUENT REFRESHES
I WANT TO LOAD THE LAST 10 DAYS
OR SO. SO WHAT I CAN DO I CAN COME
IN, AND DEFINE THIS INCREMENTAL
REFRESH POLICY. IN THIS CASE I'M
SAYING LOAD THE LAST FIVE YEARS
OF DATA AND EVERY TIME I DO A REFRESH
JUST REFRESH THE LAST 10 DAYS. SO
THE FIRST TIME IT'S REFRESHED IT
WILL TAKE A LOT LONGER. AND SO I'VE
DEFINED THIS. THERE'S A LITTLE BIT
MORE TO IT. YOU'VE GOT TO HAVE END
PARAMETERS SET UP FOR IT. I'VE GOT
TO HAVE THESE END PARAMETERS SET
UP AND THEN GOT TO HAVE A FILTER
SET UP. THE REASON FOR THIS. I DON'T
WANT TO HAVE 10 BILLION ROWS IN
DESKTOP. MY LAPTOP DOESN'T HAVE
A QUARTER OF A TERABYTE OF MEMORY.
I DON'T HAVE THE RESOURCES TO LOAD
THIS MUCH DATA ON MY LAPTOP. I DEFINE
THESE PARAMETERS AND IT'S ONLY LOAD
10 DAYS OF DATA. BECAUSE IT'S FILTERED
BASED OFF THESE PARAMETERS. HAVING
SET THAT UP AND I CAN DEFINE THIS
INCREMENTAL POLICY. NOW I GO AHEAD
AND PUBLISH IT TO THE SERVICE. BY
THE WAY EVEN WITH THE INCREMENTAL
REFRESH POLICY IF I DO A REFRESH
IT'S STILL ONLY 10 DAYS OF DATA.
I NOW GO AHEAD AND PUBLISH THIS
TO THE SERVICE. I'VE GOT IT PUBLISHED.
SO SO I'M GOING TO SHOW YOU. I'M
HERE IN THE SERVICE AND I PUBLISHED
THIS MODEL TO THE SERVICE. AND THEN
AT THIS POINT I COULD JUST CLICK
REFRESH AND LEAVE IT RUNNING AND
COME BACK THE NEXT DAY. FOR FIVE
YEARS OF DATA IT'S GOING TO TAKE
A WHILE. RATHER THAN DOING IT HERE
I'M GOING TO DO IT FROM THE ONE
OF THE MANAGEMENT TOOLS FROM POWERBI
MANAGEMENT SERVICES. AND HERE I
CAN COPY THE WORKSPACE URL, AND
NOW I CAN CONNECT TO THAT AS IT'S
A ANALYSIS SERVER. THE TRADITIONAL
MANAGEMENT TOOL FOR ANALYSIS SERVICES.
SO HERE YOU SEE I'M CONNECTED TO
THAT SAME WORKSPACE URL, SC AND
HERE I HAVE THE SAME DATABASES OR
THEY'RE THE SAME DATA SETS YOU SEE
HERE. SO THEN NOW IF I SHOW -- HERE
THE REASON I WANT TO DO IT FROM
MANAGEMENT STUDIO YOU GET MUCH MORE
VISIBILITY AND WHAT'S GOING ON INSIDE.
IF YOU JUST REFRESH FROM THE POWERBI
SERVICE IT SAYS I'M D DONE. HERE
I CAN GET ALL OF THE PARTITIONS
THAT ARE REFRESHED. SO FOR THESE
LARGE DATA SETS. BUT THE CASE OF
POWERBI. >> SO EITHER YOU'RE OLD
SCHOOL OR YOU'RE INTERESTED OR YOU
DON'T WANT TO DO THE EXTRA WORK.
YOU CAN COME INTO MANAGEMENT STUDIO.
I'M GOING TO COME INTO THE PARTITIONS
FOR THIS TABLE. SO IF I'VE JUST
PUBLISHED THIS FROM POWERBI DESKTOP
THERE WILL BE ONE PARTITION. POWERBI
DESKTOP DOESN'T KNOW ABOUT PARTITIONS.
RIGHT NOW IT'S GOT ONE PARTITION
AND IT'S GOT THE 10 DAYS IN IT.
NOW I'M GOING TO PERFORM A REFRESH.
USING THIS TYPE OF SCRIPTING LANGUAGE
COMMAND AND TO GENERATE THIS COMMAND
ON A PROCESS A TABLE. HERE IN MANAGEMENT
STUDIO YOU'VE GOT FINE GRAIN CONTROL
LEVEL. DATA SETS, TABLES, PARTITIONS,
WHATEVER YOU WANT. IF I SAY PROCESS
TABLE AND THEN I SAY FULL AND I
CLICK SCRIPT IT WOULD CREATE A SCRIPT
-- CURRENTLY IT WOULDN'T HAVE THESE
PARAMETERS FILLED IN. SO THIS ONE
JUST APPLY THE REFRESH POLICY. THAT
WOULD BE THE DEFAULT. THE EFFECTIVE
DATE IS OVERRIDING THE CURRENT DATE.
WHILE MY DEMO DATA DOESN'T GO TO
THE CURRENT DAY. JUST PRETEND THE
CURRENT DAY IS AUGUST 15th. HAVING
DONE THAT I CAN COME BACK TO THE
PARTITION DIALOGUE. IF I REFRESH
IT GENERATED A TON OF PARTITIONS
FOR ME. OBVIOUSLY THIS ISN'T --
IT WOULDN'T HAVE RUN IN SECONDS.
I DON'T HAVE THE FULL DATA SET HERE.
IT WOULD HAVE TAKEN A WHILE TO RUN.
SO I HAVEN'T DONE THAT. IT'S LOADED
FOR THE FIVE YEARS OF DATA, AND
THEN YOU'LL NOTICE IT'S GOT PARTITIONS
FOR QUARTERS, MONTHS AND DAYS UP
TO THE DATE THAT I SPECIFIED AS
THE CURRENT DATE. THE REASON IT
NEEDS THESE SO IT CAN MAINTAIN ITS
CONTRACTOR. IF IT HAD YEARS I'D
HAVE TO REFRESH THE WHOLE YEAR.
TO BE REALLY EFFICIENT THAT'S WHY
IT KEEPS THESE LIKE THIS. AS WE
ROLL FORWARD IT WILL GENERATE A
NEW DAY AND IT OPPORTUNISTICALLY
MERGE THESE AND IT'S MORE EFFICIENT.
I'LL GO AHEAD -- AT THE MOMENT WE'RE
UP TO THIS DATE. I'M GOING TO GO
AHEAD AND ROLL FORWARD ONE DAY,
AND THEN WE SHOULD SEE A NEW PARTITION
GET GENERATED. THERE IT IS. SO WE'VE
GOT A NEW PARTITION HERE. THE OTHER
THING I WANT TO SHOW IS THAT IF
I -- IT'S ABOUT HERE. YOU CAN SEE
THAT THE TIME STAMP CHANGES WHEN
IT WAS LAST REFRESHED. THERE'S A
GAP OF A COUPLE OF MINUTE. THAT'S
ME BLAB RING ON. THERE'S A TWO-MINUTE
GAP WHEN I WAS TALKING. I'VE REALLY
GOT THE HISTORY. I'M GOING TO LOAD
THE LAST 10 DAYS OF DATA. MAKE SENSE.
SO HAVING DONE THAT YOUR NEXT QUESTION
WHAT IF I WANT TO MAKE CHANGES TO
THE DATA SET AND REDEPLOY A NEW
VERSION OF IT. SO YOU KNOW IN POWERBI
DESKTOP LET'S SAY I MAKE SOME CHANGES.
SO I SELECT THE WORKSPACE, AND THEN
IT SAYS YOU'VE GOT ONE. DO YOU WANT
TO REPLACE IT WITH THIS ONE. REPLACE
IT YOU'RE NOT ONLY TO REPLACE THE
METADATA BUT ACTUALLY THE DATA AS
WELL. IF YOU'VE TAKEN FIVE HOURS
TO LOAD ALL OF THE HISTORY IT WILL
GET WIPED IF YOU DO A STRAIGHT PUBLISH
PUBLISH. >> WEDNESDAY THE LAUNCHES
IN I DON'T KNOW HOW MANY MONTHS
TWO OR THREE MONTHS. WE'RE NOT SURE
EXACTLY. ONCE THE COMX XML AA FOR
YOU TO BE ABLE TO SCRIPT OUT AND
RUN THESE SCRIPTS AND PROGRAMMATICALLY
CREATE MODELS AND DO ALL THIS KIND
OF MANAGEABILITY AGAINST THE ANALYSIS
SERVICES MODEL. YOU'LL BE EVEN WITH
THE ABSENCE OF THE METADATA PUBLISH
YOU'LL BE ABLE TO USE TOOLS LIKE
THIS TOOL. THIS TOOL IS AN OPEN
SOURCE TOOL. IT'S GALLED THE ALM
TOOL KIT IT'S BASICALLY A ANALYSIS
GIF. YOU KNOW, THIS IS SCHEMA DIF
IS A WELL-ESTABLISHED FOR DATA WAREHOUSE
DEPLOYMENTS. IT'S THE SAME CONCEPT
OF A DATA WAREHOUSE. YOU WANT TO
ADD THE EXTRA COLUMNS AND ROWS.
WITHOUT MESSING WITH EVERYTHING
THAT'S THERE. MAYBE ACROSS ENVIRONMENTS.
AND YOU CAN USE IT FOR MASTER METADATA
MANAGEMENT TO HAVE THIS SAME CONFORMED
DIMENSIONS IN DIFFERENT MODELS AND
A THINGS LIKE THAT. THERE'S A LOT
OF CASES FOR SCHEMA DIF TOOLS. IT'S
AN OPEN SOURCE TOOL AND IT WILL
WORK IN FULL WITH THE I SELECT THE
SOURCE AND THE TARGET DATA SET IN
THE POWERBI SERVICE, AND THEN IT
DOES A COMPLETE ANALYSIS. FOR ALL
OF THE OBJECTS WHAT IS DIFFERENT,
AND WHAT'S THE SAME. SO IT'S SAYING,
OKAY. THIS ONE IS A BIT DIFFERENT.
LET'S SEE WHAT'S DIFFERENT ABOUT
IT. OKAY THE PARTITION EXPRESSION
IS DIFFERENT. FOR EACH OF THE OBJECTS
YOU CAN SEE WHAT'S DIFFERENT. AND
THEN YOU CAN LIKE PICK AND CHOOSE
WHAT YOU WANT TO MESS WITH. FOR
EXAMPLE YOU'RE GOING TO DO A NEW
DEPLOYMENT YOU DON'T WANT TO MESS
WITH ANYTHING THAT'S THERE. IN CASE
YOU BREAK ANY EXISTING REPORTS.
ANYTHING THAT IS A DELETE ACTION
ITEM BECAUSE IT DOESN'T EXIST IN
THE SOURCE DATABASE. I DON'T WANT
TO BREAK ANY EXISTING REPORTS. SO
I'M GOING TO ALL OF THE DELETE ACTION
ITEMS CAN GET SKIPPED IN ONE GO.
JUST FOCUS ON THE UPDATES AND THE
CREATES. AND THEN I CAN PICK AND
CHOOSE WHAT I WANT TO DEPLOY. I'M
NOT READY TO DEPLOY THOSE OBJECTS
FOR WHATEVER REASON. I'M WORKING
ON THOSE. SO I WANT TO DEPLOY THESE
OTHER OBJECTS. NOW I VALIDATE THE
SELECTION. THIS IS THE KEY. IT'S
GOING TO VALIDATE WHAT YOU'RE GOING
TO DO ISN'T GOING TO BREAK THE MODEL.
ACTUALLY LIKE A MILLION DIFFERENT
VALIDATIONS IT DOES HERE. THIS IS
JUST A RANDOM ONE. UNABLE TO UPDATE
THE FACT BURNETT CELLS -- THERE
ARE INTERDEPENDENCIES BETWEEN THESE
OBJECTS. YOU KCAN'T DELETE I DON'T
KNOW LET'S SAY A RELATIONSHIP OR
YOU CAN'T DELETE A TABLE THAT A
RELATIONSHIP IS REFERRING TO. ALL
OF THESE ENTER DEPENDENCY IN THE
DATA SET. IT'S GOING TO DISALLOW
ANYTHING THAT WOULD BREAK THE MODEL.
HAVING DONE THAT. SO IN THE CASE
OF INCREMENTAL REFRESH I WANT TO
SAY FOR THAT TABLE DON'T MESS WITH
IT. THERE'S SOME POLICIES AS A POLICY
JUST NOT LOSE THE HISTORICAL DATA.
I WANT TO KEEP THOSE PARTITIONS.
SO I CAN JUST NOT MESS WITH THE
PARTITIONS. I CAN DEPLOY THE NEW
MEASURES. I CAN KEEP ALL OF THE
HISTORICAL DATA. AND THEN I WOULD
UPDATE THE TARGET OR GENERATE ONE
OF THOSE SCRIPTS FOR MANAGEMENT
TOAD IO AND I CAN HAND IT TO A DBA.
BASICALLY THAT'S THE CONCEPT. ALL
TO SUMMARIZE WE'RE BRINGING THE
SCALEABILITY, THE MANAGEABILITY
AND THE OPEN PLAT PLATFORM SERVICES
TO POWERBI. ANALYSIS SERVICES HAS
HUGE SO WE'RE BRINGING THAT THE
SCALEABILITY TO POWER BI. THE LARGE
MODEL WILL BE COMING SOON. AND YOU'LL
GET THE MANAGEABILITY. YOU GET THE
SCRIPTING. YOU'LL GET THE PROGRAMMABILITY.
YOU'LL GET INCREMENTAL DEPLOYMENTS.
ALL OF THE MANAGEABILITY OF MANAGEMENT
SERVICES AND THEN SOME. WE'LL INCREMENTALLY
ADD THINGS TO POWER BI. THANK YOU