Hello
The industry is talking a lot these days about augmented analytics
as the new generation of analysis capabilities
using predictive technology.
When we think about augmented reality
it’s quite natural to think about overlay images.
For augmented analytics, it’s not that easy to get a feel for it.
So today with SAP Analytics Cloud I’d like to give you a concrete sense
of what augmented analytics is, what it can do for you,
how it can boost your analytical thinking.
SAP Analytics Cloud was built from the ground up
with augmented analytics mind
For this I’ll use the College Scorecard dataset.
The College Scorecard was created by the US Government in 2015
under the Obama administration to help students decide
which college made most financial sense when considering
tuitions, loans, grants and future earnings
Calling it a scorecard is an understatement.
What the U.S. Department of Education is releasing is a dataset
of epic proportions and it’s a beautiful raw material for data analysis.
Gartner selected it for its BI Bake Off contest both in 2016 and in 2019,
the BI Bake Off takes place during the Gartner’s Data & Analytics summit.
So here we see the College Scorecard dataset in SAP Analytics Cloud
data preparation interface, before recording this video
I used it to prepare the dataset so that is can be analyzed
that’s one thing that augmented analytics has in common
with traditional analytics, it needs good data.
The most important thing is to make sure
that the dimensions and measures
are defined correctly.
I also renamed columns technical names with a more
understandable descriptions
and some dimensions that contained numeric codes,
I’ve replaced those codes with the corresponding descriptions.
The dataset is very well documented
So, each row corresponds to a college.
It provides information on the college itself
like the number of students,
the degrees awarded,
the school revenue and expenditure,
the tuition fees
the admission rate,
the students admission scores
the completion rate,
its location, is it in a big city, a small city, in a suburbs.
It also provides info on the students
their age, their ethnicity,
if they are financially dependent of their family,
if they are part-time,
are they first-generation Americans,
do they come from a poor or a rich neighborhood,
this is based on census data,
did they get grant,
do they have loans,
repayment data, default rates.
and of course, earnings after college
In total hundreds of fields for thousands of colleges across the US.
We’ll focus our analysis on the former students’ earnings
10 years after college entry.
What’s are the key factors
that will drive how much people will earn later on.
So once we are happy with how the data looks like,
we can get started with the analysis
of student earnings after college.
This is were things get awkward.
How to approach this,
where to look, what to look for, what to focus on?
We know we have hundreds of facts to consider,
I already see myself slicing & dicing, comparing,
trying to find patterns for countless hours
in the hope of finding nuggets of information that latter I can share.
Praying that I don’t miss anything crucial.
That’s what would happen with a traditional Data Discovery tool,
thankfully with SAP Analytics Cloud
what we will experience will be radically different.
Traditional Data Discovery is good for simple questions,
closed-ended questions
like “Has the debt per student increased this year?”
But it falls short for the more complex, open-ended questions.
We’ll see that actually the two approaches
complement one another, using them together
in SAP Analytics Cloud will provides us terrific value.
So here what I did is launch a feature called Smart Discovery
we are going to start by leveraging the augmented BI capabilities.
We could just go ahead hit run and ask
SAP Analytics Cloud
what is influencing earnings 10 years after college.
But before we are going to exclude some measures
that are obviously correlated with earnings 10 years after college
and that we are not interested in
these are earnings 6, 7, 8 years after colleges,
other stats on earnings like the standard deviation,
we are also excluding some facts on student loans & debt,
facts on repayment
data on student default rate, and a bunch of other facts.
We are also excluding some dimensions that are just not relevant.
Behind the scene
SAP Analytics Cloud is building an machine learning
regression model,
we absolutely don’t need to care about this.
This is were the magic on SAP’s unique automated
predictive modeling technology is happening.
On the first page we see a distribution of the earnings,
but what we are the most interested in is on the second page
It provides the dimensions and the measures
that have the most influence on future earnings.
This is what we are looking for.
We see that Carnegie classification
is by far what has the most influence
on future earnings.
The share of students over 23 and the median family income
are the second and third top influencers.
I love this
SAP Analytics Cloud
also tells us that the median family income itself
is correlated with the average family income, no surprise here
and with the share of aided students
coming from poor and wealthy families.
This totally make sense, financial aid depends on family income !
So let’s look first at the Carnegie Classification,
we see that top earnings correspond to doctorate universities
and medical schools.
Let’s check to what colleges this corresponds too.
The beauty of SAP Analytics Cloud,
the beauty of Augmented Analytics
is that it sets in our analysis, our questioning in motion.
Here we open the standard data exploration interface,
we filter on doctorates & medical schools,
switch a table,
add the college name,
replace the classification bin that was automatically generated
by the source classification dimension
and sort on the highest earnings.
We see that these are medical schools,
if we filter on doctorate universities only,
we see that theses colleges are the
MIT, Stanford, Harvard, Yale, Caltech and the likes.
We’ll come back to this in the next video,
will use augmented analytics to analyze earnings excluding
these very specific colleges,
you’ll this will bring us a wealth of new findings.
For now, let’s continue to look at the factors
that influence future earnings across all colleges,
so the low rate or a high rate of
students over 23 has a positive influence,
that probably reflects some sub-segments of colleges,
something we could explore as well.
That’s typical of data exploration,
an answer, leads to more questions.
The higher the student’s family income,
the higher the earnings 10 years after college,
that’s unexpected and alarming.
That means it’s not only student’s merits and degrees that
that determine future earnings.
Students with a bachelor will earn more,
that’s expected in somewhat reassuring.
Next is the college expenditure:
Spending per student has a positive influence on earnings
that makes sense.
The share of aided students from low income family
has a negative impact of future earnings.
In a traditional Data Discovery tool, you could easily conclude
the more the students are aided the less they are successful.
That is just plain wrong!
We just saw
that the share of aided students is correlated with the family income.
We are dealing here with a set of interdependent factors
influencing future earnings.
They need to be considered as a whole,
they need to be considered together,
AI is far superior to humans for this.
Completion rate
has positive impact,
that's expected if you get your degree you will earn more.
Earnings is the highest for colleges
where the share of women is around half,
between 46% and 53%.
What ‘s intriguing here, is that, the greater the share of women is,
the smaller the earnings.
We definitely need to dig into that further.
The next influencer is an interesting one as well,
historically black colleges & universities, HCBUs were established
during the segregation prior to the 1964 Civil Rights Act.
At that time, most colleges in the southern states prohibited
African Americans from attending.
So we see that students from HCBUs,
from historically black colleges
tend to earn less after college.
This is most likely related to another key influencer we found earlier
family income
because African American family income on average is lower.
And finally, the family income of depend students,
has also a strong influence on future earnings.
So that’s the key take away so far,
and somewhat unsettling,
out of the top ten factors driving earnings 10 years after college
four are related to family income!
In the next video we’ll deepen our exploration
with stunning discoveries
I kid you not,
and rip the full benefits of Augmented Analytics.
See you there.