PRIYANKA VERGADIA: Hi.
I'm Priyanka Vergadia, and this is "AI Simplified," where we
learn to make our data useful.
In this video, we are predicting the future.
That sounds like I'm an astrologer, but not really.
We are going to predict a different kind of future
influenced by data.
Yeah, you're right, we're covering forecasting models--
what they are and how to create them in Vertex Forecast.
But before we dive into it, what are
some of the applications of forecasting?
Say you're a retailer and you need
to predict product demand or sales,
or you're a call center manager and you're predicting the call
volume to hire more representatives,
you're a hotel chain predicting hotel occupancy
for next season, or you're a hospital and forecasting bed
occupancy.
You get the idea.
Forecasting data sets come in many shapes and sizes.
There are univariate data sets, where
we observe a single variable over a period of time.
For example, you can think about weather data or airlines data.
Even though it's just one variable,
sometimes this is all that you need to forecast
your data into the future.
You can see clear patterns, the seasonal peaks, and also
the trends in that data.
More often, we are faced with multivariate data sets,
where you have historical observations of the variables
you're forecasting and other related time series.
These are often referred to as covariates, and covariates,
because the value varies with the target variable.
These related time series can be very helpful to forecast
your target variable.
And maybe, for example, you're plotting rain versus umbrella
sales versus Uber rides.
Now, we all know how taxis can be hard to get when it suddenly
starts to rain.
So as an owner of a taxi company,
I can use the weather forecast to anticipate the higher
demand in my cars and bring the cars online.
Be aware that not all covariates are useful,
even if they correlate strongly with your target variable.
Some common examples of covariates
used in retail forecasting are holidays, third-party forecasts
such as weather, and you can have
planned promotions, foot traffic to the stores, and so on.
Up next, we are going to look at forecasting algorithms.
Now, here are the algorithms we can use for forecasting.
The most popular methods today are statistical methods,
which are also called as classical methods.
There are many types of statistical models,
including ARIMA models, that are being used for most popular use
cases.
Now, Google offers an advanced ARIMA plus model
with BQML, which is BigQuery ML.
The other class is machine learning models,
like various tree-based methods and neural networks.
Now, these deep learning models have
been gaining a lot of popularity for forecasting applications
over the past few years.
Now, obviously, there's still a lot
of debate on when to apply which methods,
but it's becoming increasingly clear
that neural networks are here to stay
for forecasting applications.
These methods perform fairly well
across a broad range of forecasting problems,
and particularly well when dealing with forecasting
newer item demands.
And these are called cold starts,
and forecasting cold starts on items with limited or varying
historical data.
Now, deep learning models cross-learn
across very large groups of related time series
with varying lengths and missing data.
The good news is that you don't really
need to know exactly which architecture to use
because Vertex Forecast does that for you.
Now, we can build forecasting models
in Vertex using advanced auto ML algorithms for neural network
architecture search.
Now, we automated the preprocessing of your time
series data, and, instead of fumbling into data types
and transformations, you can just
load your data into the data set in Vertex AI,
and auto ML would automatically apply
those common transformations and even engineer features
for you required for modeling.
Now, most importantly, we could evaluate
hundreds of models in parallel to find the right architecture
for your particular data set.
And this requires a lot of computational resources.
But because we can run all these trials in parallel,
we're still able to find the best model
and tune it to your particular data
set in the fraction of time it typically
takes for a traditional method.
And of course, we can assemble the best model
and together to ensure the model is
stable across a wide range of predicted outputs.
Now, how does all this work?
Let's jump into the console and see.
From our Vertex AI dashboard, we create a new tabular data set.
We can see there is an option for forecasting.
You may be curious if you can apply a regression
model to your time series data.
Now, the difference between them is
that regression and classification, the target's
predictive value depends only on the values of the feature
columns in the same row.
Now, in forecasting, the target's predictive value
also depends on the context values
of the target and the features.
Now, we will see more on this further in the demo.
In regression and classification problems,
the output is one value.
But as you can imagine, in forecasting problems,
the output is a sequence of values.
Now let's select Forecasting here and proceed.
We are using the Iowa Liquor Sales public dataset available
in BigQuery.
Now, it consists of wholesale liquor purchases in the United
States state of Iowa since 2012, and we
can preview this data here.
For the purpose of this demo, I have already
done some basic data preprocessing to group
the purchases by day, and we will
use a CSV extract from the BigQuery table.
Now, back in our data set, let's import the data
from our Cloud Storage bucket.
You can also upload the data directly from your computer
or BigQuery table here.
Once the data is imported, we can
configure the model features.
The time series identify column is
to specify what time series a row belongs to.
Now, every unique combination of key values
defines a different time series.
For example, here in ID is our time series identifier.
Now, if we need to forecast on multiple time
series, such as product and store levels, both of those
could be used as keys.
Now, time column is the identifier, such as date
or days, et cetera.
Y is our sum of purchases that day,
and holiday is a Boolean to indicate a holiday.
Then we can train the model.
Select the target column.
This is the value we are predicting.
So in our case, it is sales, or the Y column.
We have already set our time series identifier and the time
column, so next we need to set the date of frequency.
I'm selecting hourly here and setting the forecasting horizon
to 7.
This specifies the number of periods
that the model can predict into the future.
Now, historical context window I'm setting to 30 hours here.
This means that the model will use data from previous 30 hours
to make a prediction.
There are trade-offs between shorter and longer windows,
and generally selecting a value between 1 to 5 times
the forecasting horizon is recommended.
Now let's check the box to export
test data set to BigQuery.
We can leave it blank, and it will automatically
create a data set and table in our project within BigQuery.
Now, in the training options, we can specify more details
about how we would like to train the model.
Set these columns to available for a new forecast
because we know this information already.
Change the optimization objective to MAE,
or Mean Average Error, which is more resilient to outliers.
And because we are working with daily purchase data that
can have wild fluctuations, MAE is an appropriate metric
to use here.
Before we initiate the training, set a budget.
In this case, one node hour is sufficient to train the model.
Now, once the model is trained, we
can evaluate it and go to BigQuery console
to view the predictions on test data.
Our new data set is automatically
created where we find the evaluated data items table
to review the predictions.
This table has a couple of new columns here--
the Predicted on Date column, which
is the date that the predictions were made, and Predicted Y
column is the predicted sales value.
Now, finally, we would want to use your model
to make predictions.
We have an input file in the CSV format on Cloud Storage,
and it contains MT values for the dates
to be predicted along with the historical data.
After it is complete, we can click on the batch prediction
job to view the details, including the export location.
We sent our predictions to BigQuery,
which we can see here, and the job creates two different files
or tables.
One contains any rows with errors,
and the other contains predictions.
Lastly, now that we have our batch predictions ready,
we can evaluate our forecasts in Data Studio,
which is Google's free data visualization tool.
The blue line shows my historical values,
and the red lines are my forecast.
Now, it looks like the model picked up
on quite a bit of signal in the data.
So in this video, we learned about forecasting models,
what they are, and how to create them in Vertex Forecast.
Give this a try yourself, and send me your questions
in the comments below.