[MUSIC PLAYING]
ACE NASSRI: Hey folks, I'm Ace.
CHEN YU: I'm Chen.
ACE NASSRI: And today we're going
to be talking about how did you intelligent Gmail processing
using Cloud Functions.
So first, a quick background on what we're doing today
and how our application works.
So you probably know about the various G Suite web
apps, everything from Gmail, to Calendar, to Slides, to Docs,
to Sheets, so on and so forth.
But what you may not know about is the APIs
behind these apps that effectively provide us scalable
and programmatic access to the G Suite data underlying
these applications.
Now of course, in terms of what we can do with these APIs,
especially when we combine them with Google Cloud,
the sky really is the limit here.
Just to pick on a couple use cases in particular,
we could say maybe we want to categorize
images in our Gmail messages using the Cloud Vision API.
Maybe we want to export data we generate in GCP--
say, in a BigQuery job or something-- to Google Sheets.
Maybe we want to even find sensitive data using the Data
Loss Prevention API that we have in our Google Drive accounts.
So today, we're going to focus particularly
on that first use case.
So when it comes to Gmail processing, what
we're going to try and do is say we
want to take all of our incoming images-- or, excuse me,
incoming messages.
And anything that has an image on it,
we want to find that image, extract that image,
classify it using the Cloud Vision API,
and export those classifiers to--
excuse me-- to Google Sheets, right?
So that's what we're going to do today.
And what we can do with this extension, the sort of "so
what?" if you will, is that we can use this to easily search
for images in our inbox.
So in terms of the products and services that we're
going to use today, obviously we're going to be using Gmail.
And Gmail has this really cool feature called the Watch API.
Now what the Watch API lets you do
is send notifications to a Cloud Pub/Sub topic
whenever you receive an incoming email.
So Cloud Pub/Sub provides us simple, scalable message cues
which we can then use to trigger Cloud Functions, which will
host our event processing code.
So in turn, Cloud Functions then contacts Cloud Vision and says,
hey, I have an image.
I want you to tell me what kinds of categories
apply to this image.
So maybe, let's say, it's an image of blue sky.
So Cloud Vision is going to say, OK, well, maybe it's blue,
maybe it has the sky in it, maybe it has to do with nature.
Then, Cloud Functions is going to take those categories,
export them to a Google Sheet using the Sheets API.
So now I'm going to hand it over to Chen,
who is going to talk about how we built this application.
CHEN YU: Hi.
So this project is actually a simple five-step process.
First, you set up Gmail push notifications
so that every time a new message arrives at your Gmail inbox,
Gmail will send a push notification to Cloud Pub/Sub.
Cloud Pub/Sub will then deliver this push notification
to Cloud Functions.
Another function will query the Gmail API
and download the full message.
If there is an image attachment, Cloud Functions
will send the image data to Cloud Region API for analysis,
and Cloud Functions will then write an analysis
to a Google Sheet.
Now imagine that I send myself a picture of blue sky.
Gmail will send a push notification
upon the arrival of this message,
and the push notification will be delivered via Cloud Pub/Sub
to the Cloud Functions.
Since there is an image attachment,
Cloud Functions will send the image data
to Cloud Vision API for analysis.
This is a picture of blue sky.
So as Ace has just mentioned, Cloud Vision API will probably
return results like blue, sky, nature, et cetera,
and Cloud Functions will write a result to a Google Sheet.
Now, we would love to talk more about Gmail, Cloud Vision
API, et cetera.
But before that, we really want to mention
that accessing G Suite data requires explicit approval
from end users.
G Suite APIs, along other Google APIs,
are protected by the industry standard Open L2 workflow.
Normally, you have to perform this workflow manually
to get access, but here, we used this really convenient Node.js
lab array built by the team to help us automate the process.
Here's the code for handling the Open Auth workflow.
As you can see, this package how provides us two
really convenient functions.
Also need and else callback.
The [INAUDIBLE] need function, as its name implies,
helps initiate the Open Auth workflow.
And the callback function processes
authorization data returned from Google servers,
and save it in persistent storage for future reference.
Here's a code for downloading the Gmail message.
As you can see, the code is pretty straightforward.
You just [INAUDIBLE] the Gmail API
for the most recent messages.
Gmail API will return you a number of message IDs,
and you can use those IDs to download the full message.
Here's a code for analyzing data using Cloud Vision API.
You basically just send the image data to Cloud Vision API
and Cloud Vision API will return you a number of labels,
the categories of the image.
We here detect the top three labels
and write them to a Google Sheet.
The code, once again, is pretty straightforward.
It would simply just call the Google Sheet API,
tell the API that I have a new row of data
I need to append to the sheet, and the Google Sheet API
will perform the operation automatically for you
behind the scenes.
Now I will hand it back to Ace to talk about running
and deploy this [INAUDIBLE].
ACE NASSRI: So you'd think, given
how many different possibilities are exposed
by this combination of Cloud and G Suite,
that deploying this would actually be a lot of effort,
right?
Surprisingly, it's not.
You can generally break it down into these three simple steps.
So first of all, you're going to want
to enable these various APIs that we have listed to the left
here.
Then you're going to want to download the code by cloning
the GitHub project.
Now, the URL for this is github.com/googlecodelabs/gcf
gmail codelab.
And unfortunately we are having a couple of technical issues
with our code labs, but if and when those are resolved,
you can also check this out in the hands-on lab section
as well.
Now, last but not least, you're going
to want to follow the instructions in that code
lab, which will talk you through deploying this code to GCP.
And now I'm going to hand it back
to Chen, who's going to take us through a demo.
CHEN YU: So we start this demo by calling
the all see need function.
This function will help us set up the Gmail push
notifications.
And this function will redirect us
to the sign-in, this Google page where you can sign in
with your own Google account.
Here you will see this "this app isn't verified" warning.
That's because we are testing this locally.
So it's actually safe to ignore this warning.
And now Google will tell us that this app requires
the following permissions to continue.
Basically, it needs access to your Gmail inbox
so that it can set up the Gmail push notification.
You grant the system the access, you
will see this success message, "successfully set up
Gmail push notifications."
Now Ace will send me an email with a picture.
ACE NASSRI: Unfortunately, we're having
issues with the internet here.
But we have preferred some of these images
so you can see what it looks like in the real world.
CHEN YU: Yeah, so basically as you can see here,
here's a picture of a beautiful cat.
And this image has been categorized by the Cold Vision
API here, along with the sender of the email
and the name of the attachment.
Let's see if the--
it will probably still take a few seconds for the new message
to arrive.
ACE NASSRI: It's being quite slow.
So unfortunately our conference Wi-Fi-- hit or miss.
CHEN YU: Let me give it one more second.
ACE NASSRI: It's having issues.
We should probably just go on.
Technical difficulties, everyone.
CHEN YU: Sorry about that.
ACE NASSRI: So that's it from us.
If you have any questions, please do let us know.
Either email us.
I am faast@google.com-- faast with two A's.
You can also DM on Twitter @serverl0ss.
CHEN YU: I'm chenyumic@google.com.
ACE NASSRI: Thanks for your time,
and we hope you enjoy the rest of your time
here at Google Cloud Next.
CHEN YU: Thanks so much.
[APPLAUSE]
[MUSIC PLAYING]