In this video, you’ll see how you can visualize AWS Service Catalog product
usage with Amazon QuickSight.
With this solution, you can provision AWS Service Catalog products in each
spoke account within an AWS organization, aggregate product usage,
and then create a QuickSight dashboard to view the usage data.
This is the AWS Management Console,
where we’re logged in to the hub account for our AWS organization.
To get started, let’s navigate to AWS CloudFormation.
We’ll create a CloudFormation stack with new resources.
Let’s upload a template file from our local computer.
For demonstration purposes, we’ve already cloned a GitHub repository
containing the files we’ll need to deploy Service Catalog resources.
Next, we’ll give the stack a name.
Let’s quickly find the organization ID for our hub account.
We’ll copy the ID and paste it in the stack details.
Let’s keep “service-catalog” as the resource name prefix and continue.
Now let’s acknowledge the notice and create the stack.
We’ll wait for the stack to be provisioned.
The provisioning is complete.
Let’s take a look at the resources that were created.
We can see IAM child and parent roles,
an Amazon Simple Storage Service (Amazon S3) bucket, and a bucket policy.
Let’s upload a folder to the S3 bucket.
From within the repo that we cloned, we'll select and upload the Lambda directory.
Now let’s close this page to return to the bucket.
The bucket now contains the Lambda folder.
Drilling in, we can see a Service Catalog subfolder that contains .py and .zip files.
Now let’s return to CloudFormation to create a stack set.
With a stack set, we can deploy stacks in AWS accounts across
regions by using a single CloudFormation template.
First, we'll specify the permissions required to deploy to target accounts.
In this case, we’ll select self-service permissions.
For the IAM admin role, we’ll select the IAM parent role from the stack
we created and provide the IAM execution role name.
Next, we’ll upload a template file from the repository.
We'll name this stack set “audit-hub-stack” and continue.
Next, we’ll enter our organization ID and the primary region, which is “us-east-1.”
Let’s keep the resource name prefix.
Next, we’ll enter the name of the S3 bucket we created earlier.
Next, we’ll input the account number
for our hub account, and specify which regions to deploy the stacks to.
We'll choose US East (N.Virginia) as our primary region, followed by US West (Oregon).
These are the US East 1 and US West 2 regions.
Now let’s acknowledge the notice and deploy the stack set.
While the stack set is being created, let’s navigate to our stacks.
We can see a new stack is being provisioned.
We can also see that a DynamoDB table, Athena bucket, and Lambda
function are being provisioned as part of the stack set.
The deployment has completed successfully.
Now let's create another stack set to deploy resources to the
spokes within our hub account.
This time, we’ll choose service-managed permissions.
With this option, StackSets automatically configures the permissions
required to deploy to target accounts.
Next, we’ll upload a template file from our repository.
We'll name this stack set “audit-spoke- stack,” and enter our hub account ID.
We’ll enter “us-east-1” as the primary region.
Next, let’s configure StackSets to perform non-conflicting operations
concurrently and queue conflicting operations.
Next, let’s specify the regions to deploy the stacks to.
Again, we’ll choose the regions corresponding to US East 1 and US West 2.
Now let’s deploy the stack set.
The deployment succeeded, and we can
see stacks were created for the two regions within each spoke account.
Now let’s navigate to AWS Service Catalog.
We’re logged in to Service Catalog with one of our organization’s spoke accounts.
To test that our stack sets deployed correctly, let’s provision
one of the Service Catalog products.
We’ll provide a name and topic name,
and then launch the Amazon Simple Notification Service (SNS) product.
Now let’s return to our hub account and navigate to the DynamoDB table
that we provisioned earlier.
Let’s get a live item count to see if any items have been added to the table.
One item has been added, which confirms that Service Catalog product
provisioning from spoke accounts is being captured by the hub account.
Now let’s navigate to the AWS Serverless Application Repository
to provision a DynamoDB connector.
We’ll search the available applications for “athena federation,” and then select
the Athena DynamoDB Connector.
This launches the AWS Lamba application configuration page.
We’ll enter the name of the bucket where this function can spill data.
Next, we’ll name the Athena catalog and update the spill prefix to include “Dynamo.”
Now let’s acknowledge the notice and deploy the application.
It will take a couple of minutes to provision the Lamba function and give
it the appropriate permissions to allow Athena to query DynamoDB.
Now let’s go to Athena to add Amazon DynamoDB as our data source.
We’ll name the data source “dynamo.”
Next, we’ll select the Lambda function we just created and connect to the data source.
The data source has been created successfully,
and we can see that a default database has been associated with it.
Let’s query the database.
The default database contains the DynamoDB table that we deployed.
Before we can run our first query,
we need to set up a query result location in Amazon S3. Let’s do that now.
We’ll select the bucket that was created when we deployed the hub account resources.
Now we can query DynamoDB from Athena.
We’ll run a query to view the products that have been
provisioned in the spoke account.
Scrolling through the results, we can see the SNS product we created.
Now let’s return to AWS Service Catalog and deploy an Amazon Simple
Queue Service (SQS) product in the second spoke account we created.
We’ll generate a name, and then launch the product.
While the SQS product is provisioning, let’s provision some more products.
We’ll provide an activity name for the AWS Step Functions product, and then launch it.
Next, let’s launch the S3 product.
Now let’s go back to the first spoke account to launch the S3 and SQS products.
The DynamoDB table should now include the products we just created
in the two separate accounts.
Let’s return to Athena and run our query again.
Let’s quickly view the query results.
Now let’s go to Amazon QuickSight and build a dashboard with the data
collected in DynamoDB.
For demonstration purposes, QuickSight has already been set up.
We just need to give QuickSight access to the S3 bucket and the
AWS Lambda function used by Athena.
Let’s do that now.
We’ll deselect and then reselect Amazon Athena so that we can
specify the resources we want QuickSight to be able to access.
We’ll select the appropriate S3 bucket and Lambda function.
Notice that a new IAM role will be used by QuickSight to access S3 and Athena.
Let’s save our changes.
Now let’s create a new dataset.
We’ll add Athena as a data source.
Let’s provide a name and then validate the connection.
The connection has been validated.
Let’s proceed with creating the data source.
Next, we’ll choose the table containing our data.
We can choose to import into SPICE for quicker analytics, but in this case,
we'll directly query our data.
To create a visualization, we can select fields in the list.
For example, we can select “accountId” to query all the records for each account.
We’ll also add “productId.”
The visualization shows that we provisioned three products in each of our accounts.
Next, let’s add a pie chart.
We'll select “productName” to query all the different products that we provisioned.
Next, let’s create a horizontal bar chart.
We’ll select “productName” and “region” to see the products that
were provisioned in each region.
Since we deployed all the products within the US East 1 region,
we only see products there.
Now that we’ve finished creating our visualization, we can publish it.
We’ll name the dashboard “service-catalog-usage.”
The dashboard has been published,
and we can now share it with our team and managers.
You've just seen how you can
visualize AWS Service Catalog product usage with Amazon QuickSight.
You can learn more about this topic in the description and links for this video.
Thanks for watching. Now it's your turn to try.