[ELECTRONIC MUSIC]
KRIS BRAUN: Hi.
Welcome to this session on implementing
asynchronous operations in your UI.
I'm Kris Braun, the product lead for Workflows,
and I'll cover why getting this right
leads to building trustworthy applications.
Then my colleague Guillaume Lafore
will share a couple variations of a helpful pattern
for implementing asynchronous operations in your UI using
Workflows along with Firestore.
Your app or website exists to do the things users ask it to do.
That might be publishing their brilliant article,
placing an order, or authorizing access for a new employee.
User trust is the most valuable thing your app can build--
or lose.
How often have you experienced an app
that is flaky, unreliable, or simply
doesn't give you the feedback you need to know whether it's
working or not?
It only takes one or two bad experiences
before I stop trusting an app, which
leads me to avoid using it.
On the other hand, many apps are so reliable that they
form unconscious habits where I don't even
consider them failing to do the thing that I ask.
Our goal as developers is to create apps
that win that kind of trust.
The challenge is, many things an app can do
may not be instantaneous.
Processing may take from several seconds to several hours.
This might be because of the complexity of the computation,
waiting for a limited resource, or connection
to a real-world process that just takes time,
like 3D printing.
In addition, many operations involve multiple services
performing a chain of operations to complete the overall task.
Reliably combining these services
requires re-tries with exponential backup,
which further increases the runtime
in worst-case scenarios.
And then there's humans.
They're wonderful, really.
They can make pizzas and authorize insurance claims.
But they're rarely as fast as computers.
They often take days or months to complete tasks.
I know Guillaume had to wait longer
than he would have liked for me to finish
my part of this session.
All of these factors contribute to variable and long run times.
If any of them apply, your UI needs
to take into account the period of time the operation is
in progress rather than assuming all operations are quick
and immediately successful.
Now, while an operation is running,
a client UI might lose its network connection,
crash, or simply be closed.
If the operation depends on an HTTP request staying alive,
the real world will make these operations unreliable.
You need your operation to continue asynchronously
in the background, running on your back end
regardless of what happens to the front end.
So back to our user, who asked our app to do something.
They need to see real-time feedback to trust
that the app is working.
Few things create a more uncomfortable user experience
than clicking a button and having no idea if it worked,
or is working, or failed.
For multi-step processes, seeing which step is current
can also be important.
Today we'll show how to deliver this reliability
and reactivity.
Our pattern will use Workflows to reliably perform the work
and Firestore to drive a reactive UI.
Guillaume, over to you to walk us
through this helpful pattern.
GUILLAUME LAFORE: Workflows is a fully managed,
scalable service that can orchestrate any APIs,
internal or external.
With Workflows, you can externalize and monitor
the logic of your business processes
and have it work for you even when
your mobile application or website aren't
running or active.
The workflow is made up steps, and you can go through them,
loop through them, or create different branches.
Steps can even run in parallel.
The two cool aspects of Workflows
are that you can create callback endpoints
that service can call back into and that pauses your workflow.
And when the endpoint is called, it
resumes the workflow execution.
And secondly, Workflows can even run a super-long time,
even up to a year.
So you can support a very rich set of possible business
workflows.
For storing the status of our long-running and asynchronous
workflows, we decided to take advantage of Google Cloud
Firestore.
Firestore is a great NoSQL document database.
It's serverless, it's fully managed,
so we don't have to worry about the infrastructure management.
And it scales transparently.
It's also mobile- and web-friendly.
And in particular, thanks to the Firebase integration,
you can easily link your UI to the data
stored in the database.
The icing on the cake is the live synchronization
and offline mode capabilities.
In particular, in our use case today, the live synchronization
enables our UIs to be really reactive
and show changes as they happen in the database.
Let's see that in action with an example
showing a pizza-ordering process,
combining Workflows to orchestrate the steps
and Firestore to handle real-time status updates.
In this demo, we are going to submit some order--
for example, for a pizza.
We have several steps in our workflow.
We received the order; the payment
is going to be processed; the order is being prepared;
then the order is shipped; and at the end, it's delivered.
So you are going to receive, for example, your pizza.
We're going to store the state of our order
in the Firestore database.
So here I am seeing the documents that
are stored in the database.
And at the bottom here, we have the workflow execution.
So there were a few executions in the past.
And we're going to see that as soon as I submit the order,
a new execution is created.
And we'll have a look at the status changes in the database.
As soon as we go through one of the steps of the workflow,
we're going to update the state in the database.
And if you look at the UI, the website, mobile application,
et cetera will be notified in real time
that we went through a new step in our workflow
and we will update the UI accordingly.
So let's submit the order.
I click on the Submit button.
It is going to create a new workflow here.
So it's currently executing.
And you can see that at each step,
the status is changing-- prepared;
now it's going to be shipped.
As soon as rewriting the database,
we update the UI, turning from gray into black
and showing some ingredients.
And at the end you see that the workflow succeeded.
We went through all the steps of our workflow execution,
with status changes updating the UI live.
To recap, when the user clicks the button, the Submit button,
the client-side JavaScript calls the Google Cloud function
that triggers the execution of the workflow.
For each key step of the process,
Workflows calls and orchestrates various services
to process the payment, to prepare the order,
ship the order, et cetera.
And after each of these steps, if they execute successfully,
you update a status field in Firestore for that order.
Thanks to the Firebase SDK running in the web page
or mobile front end, the front end
is notified that a change has been made to the database.
Then the UI is updated pretty much
in real time with the status change.
On the client-side JavaScript code,
there are two key things that we need to do.
The first thing is to subscribe to real-time notifications
of the changes to the document related
to that particular order in Firestore.
When there's a change in the status data,
the onSnapshot event handler is called,
and we can update the UI via some DOM changes.
The other important thing to do is
to call the Cloud function, thanks to the fetch method,
to trigger the execution of the order-processing workflow.
The client-side JavaScript code invokes this Cloud function,
whose job is to trigger the execution of the workflow.
For that purpose, we use Workflows Execution npm Module
and its createExecution method.
We pass the order ID as a parameter to the workflow.
But we should also send the details of our order,
with the ingredients of the pizza, et cetera.
Let's have a quick look at the workflow definition showing
the two operations we combined for each step.
First of all, in this extract, a step
calls the service that prepares the order.
And the following step updates the status in Firestore.
Interestingly, updating a field in the document
in the Firestore database is really just a declarative goal
in the workflow definition.
No need to write actual database handling code.
We rely on Firestore's APIs that Workflows can call
in that declarative manner.
Here we're actually using a special connector
for Firestore.
We're not calling its REST API manually directly.
It simplifies the usage of the API.
We mentioned that humans can be part
of those long-running processes and may
need to validate, authorize, give their go, no-go agreement.
So how do we handle those situations?
That's where we'll benefit from another Workflows features,
HTTP callbacks.
I'd like to show you another demo, with a smart expense
report application where a manager needs
to approve the expense report sent by their team.
In such an application, we need someone to take a decision
and approve or reject the reports.
Let's have a look at an expense report application.
We have two views.
The first is the employee view.
The employee is going to submit receipts,
and it's going to create an expense report.
The manager here on the right--
we'll go into the administration page--
and the manager will wait for incoming expense reports.
The incoming reports will be here in the queue,
in the waiting-approval queue.
On the left, the employee is going
to select a few receipts-- so let's
say, this one and perhaps this one.
The two receipts are selected.
Now let's submit the report.
So we're going to go through different steps
of our workflow.
Submit the report.
Receipts are added.
Receipts are sent.
The report is now being processed.
So we saw a couple or three steps very fast in the UI,
because the UI is updated pretty much in real time
as soon as there's a status update after one
of the steps of the workflow.
So now the report is being processed.
We are using Document AI to parse receipts.
It's going to look at the information in those receipts--
all the line items, the amount of each item, the total,
the currency, also the location where it's been--
if it's a restaurant, et cetera.
We'll have some interesting information
that the manager will have to have a look at
and say, OK, this is a legit expense report.
I will approve or reject this expense report.
So let's wait a little bit while it's
being processed by Document AI.
Look, it appeared.
So we now have the expense report.
Notice also there were actually two real-time updates
or changes in those two UIs. There's the report that appears
in the manager's queue, and we also
have the new state, the new status here, saying report
is awaiting validation.
So now the workflow is posed and is waiting
for the manager's feedback.
And for that purpose we're going to use
a great feature of Workflows, Workflows Callbacks.
So the UI for the manager is going to call back the workflow
to say, OK, here's my decision.
I approve or I reject this expense report.
Document AI gave useful information--
the two restaurants, the items, the total, the currency,
et cetera.
So we've got all the right information.
And, well, it seems that my employee
was traveling, so he had some expenses at a restaurant.
So that's great.
I'm going to approve the expense report.
I approve it, and look-- it disappears from my queue.
And as an employee, I see report validated.
The manager has validated my report.
So I had status updates at each step of the workflow.
And the workflow was posed, waiting
for the validation or rejection by the manager,
using Workflows callback.
Here's a recap of what's happening under the hood.
The employee uses the website to store the receipts
into Cloud Storage, thanks to the Firebase SDK.
The app triggers the workflow execution
via the Cloud function.
The workflow calls the Document AI service
to figure out the various line items
of the receipts, the total amount, the currency,
the vendor, et cetera.
All those receipt details are stored
in the Firestore database.
Then in the management view, the manager
will see a new expense report arrive, ready for their review.
That's where the callback system comes into play.
The workflow is posed, waiting for the decision
of the manager.
The manager sees all the details about the expense
report and its receipts, and can then click Accept or Reject.
When either of these buttons is clicked,
then the admin UI is going to call the callback
endpoint via another Cloud function,
and that will resume the execution of the workflow.
The acceptance or rejection decision
is stored in Firestore.
And again, thanks to the live synchronization feature,
the UI will be updated live for the employee at the same time,
and they will see in the report if the report was
accepted or rejected.
We, again, combined workflows for reliably executing
a process, even waiting for a human intervention,
updating the database, and notifying all the stakeholders
live as changes are made.
Let me highlight the two key steps in the workflow that take
care of the callback mechanism.
The first step here creates a callback endpoint.
So that means there is a special URL that
has been created that is going to be open and be reachable
so that some external service can call back when it's ready,
when it's done.
Then the second step is where we pause
the execution of the workflow to wait for the call
to the callback endpoint.
How do we take advantage of this?
Well, from the UI, when the manager
clicks on the Approve or Reject button,
this callback endpoint is going to be
called, via Cloud function, with the outcome of the decision.
And then the workflow can resume its execution.
Of course, depending on the approval or rejection status,
you can have different branches of execution in your workflow
and do further steps conditionally.
Although it's not shown in these two
extracts of the whole workflow, at each key step
of the process, we are saving the status
of the process in Firestore, like in the previous demo,
so that the UI updates in real time
as we progress through the workflow
and the various statuses.
From the front end, we call the Cloud function
that will be responsible for calling the callback endpoint.
So here in this admin.js JavaScript file,
we are calling that Cloud function,
passing the callback URL we want to reach,
and also the approval status--
if that expense report has been accepted or rejected.
In the Cloud function, on the server side,
we relay the call to the Workflows callback endpoint.
We need to authenticate to call the callback endpoint;
hence why we're using the GoogleAuth
library to get an access token.
Then we call the actual URL of the callback endpoint,
passing the access token as the bearer token and the status
as payload.
Now back at you, Kris, for the last step
of the workflow of our talk.
KRIS BRAUN: What Guillaume has shown
us here is a very useful pattern for trusted apps that
can be applied widely.
Workflows ensures resilience to failures in your front-end app
and back-end services.
Operations that take time for processing, retries,
and actions are easily incorporated.
You can rest easy knowing Workflows is keeping things
running as expected.
If any back-end services fail, your workflow
can specify exactly what should happen,
and you can observe the details to remediate any failures.
Firestore handles all the magic required for your UI
to display the real-time status of the operation
as it progresses.
Operation status is current even if the page is reloaded
or the app is restarted.
Together, Workflows and Firestore
deliver the reliability and reactivity
your app needs to win users' trust.
We look forward to seeing what you'll empower your users
to do through this pattern.
Thank you for watching.
Guillaume and I are always happy to talk
about serverless application architecture,
so please reach out with any questions or ideas.
Happy building.
[ELECTRONIC MUSIC]