[MUSIC PLAYING]
CHRISTIAN SCHALK: Hello, and welcome to the Building
With AppSheet series, where I discuss building apps
with AppSheet and Google technologies.
I'm Christian Schalk, a Google Developer Advocate,
and in this episode, we'll show how
to use machine learning and AI in AppSheet.
Have you ever wanted to command your app using your own voice,
or allow your app to auto detect and insert text from images,
or let your app make predictions based on your own data?
It turns out that machine learning and AI can enable
all these behaviors and more.
Before continuing, though, here's
a bit of background on machine learning and AI.
Machine learning is a branch of artificial intelligence
that uses computer algorithms to learn and automatically
recognize patterns in data.
The data itself can be varied, but is typically text, audio,
or visual in nature.
The Google Cloud AI platform is Google's solution
for all the common branches of ML AI.
Machine learning and AI have numerous branches
with a wide array of capabilities,
but the most common branches of them
include natural language processing, which
is to understand human speech and text;
vision, to identify and process objects or artifacts in images
or video; and data classification and prediction,
to automatically classify or predict data patterns
based on sample data.
It's important to note, however, that using these technologies
typically requires significant software engineering skills
and training.
Fortunately, machine learning and AI
is simplified and streamlined via AppSheet's Intelligence
Features, which include Smart Assistant, which
uses natural language processing to enable app control via voice
command, OCR models to detect text
and images, predictive models for automatic data
classification and prediction.
Let's step through each of these features in more detail.
AppSheet's Smart Assistant uses AI speech recognition
and natural language processing to allow
users to interact with your app by using their own voice.
Simply click on the microphone icon and begin speaking.
Show me all the tasks that are due this month, for example.
Let's have a closer look at this feature in AppSheet.
OK, so here we are in AppSheet, and before we can use the Smart
Assistant, I'm going to go ahead and double check
to make sure that we have the Smart Assistant enabled
for this app, and we do.
So now I can go ahead and click on the menu
option for the Assistant, and that will give us
the microphone over here.
So I just click on that and then I can start using it.
Show me all the tasks that are due this month.
All right.
Show me all the tasks that are priority one.
Show me all the owners in a gallery view.
Show me all the owners in a map view.
All right, so that all worked fine.
Let's try another app.
OK, so here's the AppSheet National Parks sample app,
so let's try a few things here.
Show me a map of all the national parks.
Just the ones I've visited.
And just the ones within 500 miles.
So as you can see, I can add filtering
to narrow down the results.
I can also go to a direct record.
So let's try something else.
Go to Sequoia.
And there we go.
So I can do directional navigation
and I can add filters and such.
So I'll leave the rest of this feature for you
to explore in the various applications or on your own.
Our next ML AI feature in AppSheet
is OCR Models, or simply OCR, which
allows for auto detection of text and images
for rapid data entry in forms.
An automatic badge reader is a common example.
To use this feature, you first train your app
to recognize text in your images.
Once trained, your app will then automatically
recognize text from images and auto
fill corresponding form input fields.
Building an OCR Model involves the following steps.
Creating a data table with text columns
you wish to detect from an image.
The table will also need an image column
to serve as the source image to detect the text from.
You then need to add a few sample records.
Once your sample data is prepared,
you can then create a new OCR model, where
you select the table that you just created, you set the image
column, and then select the Output column where
you wish to save the data into.
Click Save to then train your app.
Upon training completion, you can then
review the image patterns found.
They should correctly identify text from your images.
You can then review the Output column's Initial Value setting.
You'll see an OCR function which sets
the fields with the identified text from the source image.
If everything looks good, you're all
set to begin using your new OCR model.
Let's walk through a real OCR example in AppSheet.
OK.
For this next demo, I'm going to show
how easy it is to build a simple badge scanner application using
the AppSheet OCR Model's Intelligence feature.
So as I mentioned, to get started,
we have a basic sheet here, which
contains the columns that map to the different fields
on the badge itself.
I also have an important image column,
which will store the image from which I will
be detecting the text from.
All right, so what I've done is I've actually taken the sheet,
pulled it into AppSheet, and generated a basic app.
So the app actually works fine, except for it
doesn't have the behavior to automatically insert
any text from the image itself.
So if I pull up an image here, none of my other text fields
are autofilled.
So to add that behavior, all I have to do
is click on the Intelligence tab and click on OCR Models,
and I'll click here to Create a New OCR Model.
As you can see, it's already filled
out the necessary options for me.
So I have the Table, the Image Column, and the Output Columns.
Now, I don't actually need the Computer Columns here,
so I'll go ahead and remove those, and I'll click Save.
And as it's saving, it's actually
training my app to be able to recognize the different text
fields and the badges.
So as it comes back here, it'll show me a couple of patterns
that it has found in training the application, right?
So I'll click on some of these patterns,
and I can see that it is actually correctly detecting
the text fields on the images itself.
So this all looks good, so now I can try my app again.
And let's go ahead and add a new record,
and I'll go ahead and just pick one there.
And there we go.
I have all the fields that have been auto filled for me.
Incidentally, if I happen to have a handwritten image, so
if I have a badge that actually has handwriting on it,
it will also be able to detect that as well.
So as you can see here, it's actually
auto detected those fields.
In this case, I don't have a Twitter handle,
so it just ignored it, but everything else looks fine.
But hopefully this gives you an idea
of how simple it is to use the AppSheet OCR Model's
Intelligence feature.
Finally, here's a quick overview of predictive models, which
allows for you to create custom machine learning models
in your apps to make predictions from.
Example use cases include categorizing customer feedback
and sentiment, or estimating the cost
of an item based on data of similar items and their costs,
and so on.
Building predictive models involves the following steps.
Similar to the OCR Models, you start
with a typical data table, ideally
with a large amount of sample data.
The more data you have, the more accurate your results will be.
Once you've created your data table,
you then create a new predictive model
where you specify the training data table,
you set the column to Predict, then select your model input
columns, which is typically all of the other columns
aside from the Prediction column.
Click Save to train your model.
And upon training completion, you
can then review the training status and model performance.
Note you can always add more sample data
and retrain to improve the accuracy.
To use your predictive model, you
check the automatically set initial value setting.
This will insert a Predict function
into the Prediction column's initial value setting.
Let's now walk through a predictive model
example in AppSheet.
OK.
For my final demo, I'm going to show
how to use predictive models to create a mobile phone price
calculator app.
And so as before, I'm starting with a basic sheet which
contains some training data.
So in this case, I have different fields
that represent different mobile phone
features, such as the battery life,
the 5G, the megapixels of my camera,
RAM, talk time, et cetera.
And then on the far right here, I have an estimated price.
So this is essentially the training data
that the app will use to be able to essentially provide
a predicted estimated price in the actual application itself.
So to get started, what I've done
is I've actually brought this sheet into AppSheet.
And as you can see, I have now the columns
that match up with those fields.
And also, I've done a little extra.
I've added some initial values here.
That makes it a little bit easier to enter in some data,
as well as I've added sliders to the different data types,
all right?
So now if I were to add a new record in my app,
I can change some of the different criteria.
I can change, like, megapixels on the camera, et cetera.
But the thing that I don't have is I
don't have the estimated price auto
updating anytime I change any of the criteria, right?
So that's the behavior that we want.
So to do that, all I have to do is go back over
to Intelligence, click on the Predictive Models,
and let's create a new predictive model, right?
So there we go.
It's creating essentially a new predictive model
that's based off of this mobile training data.
That's the sheet I just showed.
And it's correctly selecting the Estimated Price column
to Predict, and then the other fields
are the ones that will be used as model input.
All right, so now I'm going to click Save
and it's going to kick off a training process.
And the process will take, depending on how many records
and how many fields I have, anywhere
from a couple of minutes to, actually, somewhere around 45
to an hour.
It really depends on the data.
OK.
Our model is now trained, as you can see in the model summary
here.
You can see the status, how long it took to train,
as well as the performance.
So in this case, the model is predicted
to have an estimated price within $65
of the actual correct value, which is actually
pretty decent.
So in order to use the model, all I have to do
is click this option here to automatically set
the prediction initial value.
And what that does is that will insert this function
here, Predict, on the actual Estimated Price column.
And so I click Save.
And to test it, all we have to do is add a new record.
So as we add a new record to our phone calculator,
you'll see right away that we have an estimated price.
And if I start toggling some of the values here, such as RAM,
you'll see right away that it does affect the estimated price
very quickly.
So as you can see, in just a matter of a few minutes,
I was able to create a fairly reliable mobile phone
predictor based off of some training data
that I started with, and it was actually quite easy.
This concludes the Using ML AI in AppSheet" episode.
Stay tuned for more coverage of this important capability
in future episodes.
And for more info on AppSheet, Google Cloud, and Google
Workspace, check out the links in the description below.
And for more videos like this, make
sure you subscribe to this series so you don't miss out.
As always, thanks for watching.
[MUSIC PLAYING]