FILIP KNAPIK: Hi.
I'm happy to present new updates and exciting announcements
for serverless workflows in Google Cloud.
My name is Filip Knapik, and I'm a product manager
working on workflow products.
Workflows is a very broad term.
Let's us first look into some use cases
and needs around them.
Let's imagine an e-commerce application,
a workflow that needs to orchestrate a couple of steps--
for instance, first, create an invoice, then generate a PDF,
and then send it to a customer over email.
You want a workflow product, not only
to manage and orchestrate execution of those steps,
but also, for instance, recover from errors
that those steps may encounter.
Another use case that you may want to manage with a workflow
product is processing of a batch of transactions or events
or customer records, and then for each of those,
you may want to look into the contents
and make a decision on whether to take
a path with additional steps-- like, for instance, you're
looking into business transactions,
checking if they're overdue, and if so,
sending a reminder to affected users or customers.
But workflows don't have to be focused
on business transactions only.
They can also help in automation of IT landscape.
Like, for instance, you want to start a virtual machine,
then wait for some time, check whether specific conditions are
met-- like, for instance, application is up and running--
and if so, take an action, and if not take another action.
And obviously, there is many other use cases
in IT automation space and others,
and workflow products are expected
to be very flexible in automating work
across different types of endpoints and APIs.
Now, what do all those use cases have in common?
There is a set of foundational needs
that a product needs to support in order
to enable implementation of a workflow in a production
environment.
The first and probably most important
element is reliable execution of steps.
Again, in the business scenario, in the production deployment,
we can't afford to fail an execution of a workflow
just because there was a temporary network outage,
and a single step was affected, and a single step
wasn't properly executed.
And as such, a workflow product needs
to be able to recover from some of those errors,
and handle exceptions or retry steps as needed
to maximize the chance of successful execution.
Another feature that a workflow product
needs to have is efficient handover of the work.
And it's not only handover of the work and orchestration
of steps.
What a workflow product also needs to provide for
is transition and handover of the data between those steps,
because individual steps can produce data
that other steps may rely on in their execution.
The decision execution is another aspect
that is important for workflow products.
Imagine that you're building a workflow that is processing
your orders, or orders from your customers,
and you want to execute a different path if an order
exceeds a specific threshold.
Let's say beyond or above $1,000,
you want to assign a sales engineer or sales
specialist to that customer to assist them,
because the deal is so big.
And in order for you to implement such decisions
as part of your workflow directly,
you want to have a feature that allows you to manage execution
in a different way, provided that specific conditions
are met.
Workflows don't always integrate APIs or products
of the same type.
You may want to orchestrate work across different products using
different API languages, using different HTTP
methods, different authentication models, or even
host it in different places.
And a workflow product needs to be
able to integrate them all, regardless of their diversity
and differences in implementation.
Last but not least of the features
that I would like to call out is a holistic perspective
on execution and management of your workflows.
First of all, imagine that a third step of a workflow fails.
You need to be able to understand
what was the business transaction that this step
failed in to be able to recover from it,
or you may also need to see how many workflow executions
you had for a specific customer, or in a specific domain.
All those features are very important for a workflow
product.
And when looking into those, we have
realized that our customers would greatly
benefit from an introduction of a serverless workflow
orchestration product as part of Google Cloud.
And with that, I'm very happy to announce a new product
in Google Cloud family, Workflows.
What this product is expected to address
is orchestration and integration of different types of APIs.
First of all, integration of serverless compute products.
Let's say you want to create a workflow consisting
of different cloud functions, and you
want to effectively and efficiently handle
the work over in between them.
This is a great use case for Workflows,
or the same for Cloud Run.
But we don't want to stop here.
You can also create a workflow that, for instance, reads
the data from Firestore, or pushes a message to Pub/Sub,
or doing any other operation in other Google Cloud APIs.
And this is also not where we want
it to stop, because we understand
that our customers are running in hybrid environments,
and often rely on, for instance, software as a service APIs,
hosted externally to our cloud, and therefore,
Workflows products also support direct calls
to external APIs as part of the workflow definition,
including all of the handling of errors, retries, and everything
around it.
So this product is expected to help our customers
in orchestration of work across, practically speaking,
any of the APIs within Google Cloud or external,
and integration of those APIs in a holistic way
to enable implementation of workflows.
The product is fully managed and fully automated.
We want our customers to focus on their business
logic of the workflow rather than focusing on the underlying
infrastructure.
Customers are not expected, in case of Workflows,
to look into the storage or think about patching
of the underlying components.
This is all taken care of.
Another element that is fully taken care of is the scale.
Regardless if you need to run one workflow
execution per week, one per day, or 1,000 per second,
we got it covered.
The product automatically scales, and it also scales down
to 0, which means that if you're not
running your workflows for some time,
you don't have any executions, your cost
is also going down to 0.
We assume that some of our customers may want to use
Workflows in latency-sensitive workloads--
for instance, having a customer, having
a user waiting on the other side of the screen for an outcome
of a workflow execution.
To support those cases, we have engineered the product in a way
that step transitions are very quick,
so that we don't introduce unnecessary latency
in the execution of a business workflow.
We have also taken care of time it
takes to start a new workflow execution,
and there is no so-called "cold start."
So what that practically means is that starting a new workflow
execution is not only quick, but also
predictable and deterministic.
The product comes with built-in security features--
first of all, you cannot make unauthenticated requests,
so there is no risk of somebody--
for instance, a bad actor, or somebody
accidentally calling out your workflow
and starting the execution when not expected.
The product also comes with full integration
with identity and access management
of Google Cloud, which means that you
can use it to orchestrate the work of other Google Cloud
products with built-in authentication,
without worrying about tokens and other elements related
to authentication to those.
If you need to make requests to external APIs
and you need to manage secrets or passwords accordingly,
you can use Secrets Manager for this integration,
and make secure calls to external APIs, as well.
And obviously, the product comes with encryption at rest
and in transit, provided that you are using HTTPS endpoints.
By now, you are probably wondering,
how do I model a workflow?
Can you show me an example of how to do this?
And here it comes.
On the left-hand side, we're seeing
an example of the workflow, which
consists of first a call to a cloud function,
and then based on the outcome of that called function,
we're checking a condition where a transaction
number is greater than 100.
And if so, we want to trigger an execution of a Cloud Run
deployment.
Otherwise, we do not.
The right-hand side is showing the source code
that implements this specific workflow in our product.
So the first step is making a call to a Cloud Function
as part of HTTP GET request, and we're also passing some inputs
to that specific function.
We're also saying that we want to have
the outcome of that step start in metric result variable.
This is a workflow variable we're going to use later on.
In the second step of the workflow,
we have a conditional statement, which checks whether a value--
in this case, transaction number,
which is an outcome of the first step--
is greater than 100.
And if so, we're making a call to our Cloud Run deployment
using HTTPS POST method.
And by the way, we're also passing the outcome
of the first step to the second step as part of the call
to this Cloud Run deployment.
So this is a short example, but this is just
to show you how you can model a workflow using our syntax,
using our language.
We took a lot of effort to ensure that this language is
as brief, as clear to read and write in as possible,
to give the maximum benefit to our developers.
Let's have a quick look at a demo of our product.
Let's imagine that we want to build
an e-commerce workflow that sends an invoice to a customer.
The input to the workflow will be an ID of a user
that we want to perform this operation for.
The first step of the workflow we'll
be reading a record for that specific user
from our Firestore database.
If the amount due equals to 0, we
are finishing the execution at this point,
because there is nothing to do.
Otherwise, we want to generate an invoice
and send it over as an email.
We're going to use a Cloud function to generate an invoice
as a PDF, and then we want to make
a request to an external service to send an email.
Since this is an external service,
we first need to retrieve a secret
to be able to authenticate to that service.
Let's see how this could be implemented in our product.
The source code of the workflow is a YAML structure.
In the first step of the workflow,
we're making a request to Firestore API,
passing an argument, being an ID of a user,
and reading it from the database.
This obviously needs to be an authenticated request,
as otherwise, Firestore wouldn't let us read anything,
and this is what these two lines of the workflow code to do.
Whatever the output is of that step
is stored in a user record variable.
In the next step of the workflow,
we're going to check whether a specific field in the response,
amount due, equals to 0.
Now, the response of the first step was a JSON string,
and workflow engine is smart enough
to convert that into a dictionary
that we can use to validate whether the amount matches
our expectations.
If this condition is met, the workflow the execution
finishes, with no amount due message.
Otherwise, it proceeds to the next step,
being a function call using a POST request,
to our PDF generator function.
Again, this is an authenticated request,
because we don't want our function
to be executed by anyone.
And as a request body, we're also
passing the contents that we want to have in our PDF
so that our function knows what to generate.
The output of that request is stored in a PDF output
variable, which will actually be the contents of the PDF file
that we want to use as an attachment.
In the next step, we're making a request with Secrets Manager
API to read a specific secret, in my case, called email key.
And then we're going to use that secret in the request
to our email sending service, and this
is what the last step of the workflow does.
And as you can see, we're also including here an attachment,
being the PDF output body, which is
the response of the function generating our PDF.
And as the very last step of the workflow,
we are responding with a message,
saying that the user was sent an invoice for a specific amount.
So with that, let's test it out, and let's see
if it actually works.
Let me first show you the entry for a user we're
going to be testing it for, which is smith.j.
Apparently, he has $784 amount that we
need to send an invoice for.
And by the way, if you're interested,
the function that we're using to generate the PDF
is based on an open-source package
available as a PyPy module.
So let's now execute this workflow and see if it works.
So as an input, I'm passing an ID of our user, smith.j,
and I'm requesting an execution of the workflow.
As you can see, execution state is active,
but let's refresh and see if it's done.
Apparently, it is.
So the output of the workflow is the user
was sent an invoice for $784, which is what we expect,
because this is what we saw in the database.
Now, I also happen to have an email account for that user,
and as you can see, we actually got an email
that we were expecting with a PDF attachment.
Let's have a look.
And the value matches our expectation,
because this is exactly what we saw in the database.
So with this short demo, you are able to see
that we are able to orchestrate the work of different Google
Cloud products, including Firestore, Cloud
Function, or Google Secrets Manager,
as well as making a request to an external service
to send our email, and orchestration
of all of those tasks was performed flawlessly
by Workflows.
Let's now have a look into the pricing model.
First of all, we wanted to design a pricing model that
follows the promise of serverless scalability, which
means we want the cost of using the product to grow exactly
as the usage grows.
In other words, the moment you go down
to zero because you have an idle period,
or you don't need to execute a workflow for whatever
business reasons, we don't want to charge
for this specific scenario.
And therefore, the more you run your workflows,
the higher, obviously, the cost will be,
following exactly your demand and following
the growth of your business.
We differentiate two types of steps in our pricing model.
The pricing is based on number of executed steps
in a workflow.
Internal steps relate to all the elements
you can call within the Google Cloud,
like Cloud Functions or Cloud Run
or internal variable assignments,
or expression evaluations within the Google Cloud workflow
product itself.
For all those operations, the cost
is $0.01 per 1,000 steps executed.
For external requests, which are all the HTTP
your requests going to the outside world
to non-Google domain names, the cost is 2 and 1/2 cents
per 1,000 steps.
For both of those types of steps, there is a free tier.
So if you just want to give it a try,
you want to see where this product is capable of,
or you are a hobbyist and you just want to try it out,
you can use a free tier and basically get experience
from using the product, and then decide
whether you want to move on with further implementations.
So the free tier is 5,000 internal steps per month,
or 2,000 external steps per month.
With this, I wanted to very warmly invite you
all to our Beta that is available right
now to our product to start using it.
Let us know your feedback.
Let us know what features you like about this product, what
features you don't like, or what features
you feel like are missing or are preventing you
from using this product to a bigger extent.
We're very keen to listen and hear your feedback,
and I'm looking forward to more interactions.
We are also very committed to this product.
We are going to launch it to general availability
by the end of 2020.
With this, I wanted to thank you very much for attention,
and wish you happy development of Google Cloud workflows.
Thank you very much.