SPEAKER 1: In the past few videos,
we've talked about how different recommenders can
help with cost, performance, and security in your Google Cloud
projects.
For enterprises and larger organizations,
it's likely that you'll have many projects.
In order to take full advantage of the benefits
from recommendations, you might need
to scale how you handle them such as incorporating them
into a pipeline to programmatically manage them.
Today, let's go over some tips for how an enterprise could
leverage recommendations at scale
and show you how to export your recommendations to BigQuery so
that they can be reviewed across your organization.
The benefits of Active Assist were great even if you only
have one project.
But they really begin to shine in an organization with lots
of projects.
In each project, various recommenders
will help you apply the principle of least privilege
for security, minimize resource waste from workloads,
and maximize the performance of business critical applications.
Because of the increased scale of operations
in a large organization, you'll want to review and take
action on recommendations with minimal administrative work
in time.
What this looks like for your organization
will be unique to your workloads, infrastructure,
security, and processes.
While the exact details depend on what works best for you,
we recommend a three phased approach
for adopting recommenders at scale.
As you move through these phases at your own pace,
the potential for automation increases.
Let's start with the first phase-- accessing
recommendations through the recommendation hub
in the Google Cloud console.
The recommendation hub is the simplest way
of accessing and managing recommendations
in your projects and organization.
Reviewing recommendations in the recommendation hub
will familiarize you with different recommenders
available so that you can think about how you might
like to prioritize them when building
a pipeline in a later phase.
Working from the console also includes
being able to set up a BigQuery data
transfer to export your recommendations to a BigQuery
data set.
When you do this, two tables will
be generated-- recommender-export,
where the recommendations are exported;
and insight-export, where the insights that
help you focus on important patterns are exported.
Insights can provide additional context to recommendations,
or they can be used independently.
By exporting recommendations to BigQuery,
you're able to look at details for project level
recommendations across your organization.
With that ability, you can expand by creating dashboards
to help you visualize and report the potential impact
of applying recommendations.
Once you're familiarized with the console
and with the data exported to BigQuery,
the next phase of recommenders at scale is to use the APIs.
All of the recommendation data that you
can access to the Recommender hub or the BigQuery export
is also available via an API.
The recommendation resource returned by the API
will include lots of information about a single recommendation
including the priority, the expected impact,
and the status of the recommendation such as active
or dismissed.
You can also use an API to return insights.
Just like in the BigQuery exports,
insights can provide additional context
for associated recommendations.
For example, a recommendation from the IAM policy recommender
to reduce the permissions for a user
may have an associated insight that
includes the permissions that have been used in the past 90
days, and therefore should not be removed.
You can use the API to view recommendations and insights
as well as to update their status if you
choose to take action on them.
When getting started with the API,
you'll probably want to require manual review.
But you can start introducing automation
to help scale and handle more recommendations.
This can also help lay the foundation
for the third phase--
integrating recommendations into your organization's DevOps
pipelines.
If your organization has a well-established DevOps
strategy, changes to your Cloud Infrastructure
might be managed through continuous integration
and continuous deployment processes.
The goal for scaling recommendations
should be to automate the review, analysis,
and implementation.
For example, you could set up Cloud Run
to regularly retrieve recommendations via the API
and then automatically generate a pull request
in your infrastructure as code repository for that recommended
change.
That pull request can be reviewed
by an administrative team.
And if accepted merged, the included recommendation
can move on in the pipeline and subsequently
be applied to your Cloud infrastructure.
By incorporating recommendations in your CI/CD pipeline,
you can proactively monitor recommendations.
Applied changes that are kept consistent in code
can easily be reviewed, tested, and rolled back
using the same processes used for the rest
of your infrastructure.
This level of automation and integration
is ideal for large organizations and enterprises.
But no matter what scale you're working with,
you can still always take advantage of recommendations.
It can also be helpful to start slow and take the time
to get familiar with recommendations
and how they impact your Cloud infrastructure before you
implement a full blown automation process.
We've taken a look at the recommendation hub before.
So now let's take a look at how we
can get those recommendations into BigQuery.
You can do this by setting up a BigQuery data
transfer with the recommendations as the source.
To do that, you'll need to have the recommendations exporter
role assigned on your organization's IAM policy.
This will give you access to the organization's recommendations
across projects.
Start in the recommendation hub where you can find the Export
button at the top of the page.
The first step is to provide the destination project
where the recommendation data will be stored.
The destination project can be any project
that you have access to.
But you'll need to have the following permissions
for that project--
bigquery.datasets.create, bigquery.transfers.get,
bigquery.transfers.update, resourcemanager.projects.update,
serviceusage.services.enable, and serviceusage.services.get.
In addition, the BigQuery API and BigQuery transfer API
will need to be enabled for that project.
The next step is to configure the transfer
with a few details like name, schedule,
and a target BigQuery data set.
When it comes to choosing a schedule,
you'll have the option of daily, weekly, monthly,
or your own custom schedule.
You'll also be able to configure notifications via email
if the transfer fails or publish to a Pub/Sub topic
when a transfer completes.
Once complete, you'll be able to see your new BigQuery
data transfer details in the console.
And it should begin populating data within about two days.
Once the transfer has had a chance to run,
you'll see those two tables we mentioned earlier
in your selected BigQuery data set--
recommendation_export and insight_export.
We'll include a link to the documentation
where you can see the full table schema.
Recommendations will also have a field for associated insights
so you can look up any insight details in the insight exports
table.
With recommendation data exported to BigQuery,
you'll be able to write custom queries
to analyze recommendations.
From there, you can expand by exporting
to Data Studio or Google Sheets, where
you can further analyze for patterns and opportunities
to optimize.
You can also begin to automate by accessing that Recommender
API and follow that up by including recommendations
into programmatic processes as part
of your established CI/CD pipelines.
Recommendations are available for all Google Cloud users.
If your organization has a support package,
you'll also have access to the full API and BigQuery export.
We'll include a link in the description
with more information.
We'll also include some links in the description
if you'd like to find even more information
and guidance on using active assistant scale.
And remember, the right solution depends on your organization
and what makes sense for you.
Don't be afraid to implement these over time
instead of jumping immediately to the most
complicated solution.
Stay tuned for more videos.
And thanks for watching.