- Hello everyone.
This is Nishchai,
I am a Specialist Solutions Architect at AWS.
We are gonna see how to simplify data sharing
using AWS Lake Formation and as well as AWS Organization
along with the demo.
Data sharing;
what does it do in a typical, good size organization?
You are going to have multiple business unit.
This business unit operate entirely
as independent businesses.
They have their own P and L,
and most importantly,
they have their own AWS accounts.
And it's common that business unit would want to share data.
So the Business Unit 2 wants to share data
with Business Unit 1,
and Business Unit 1 would want to share back the data
to Business Unit 2,
and as well as Business Unit 1 wants to share data
with other business unit as well.
So they would want to share data with each other
but the key is they want to hold the ownership
of their data,
which is shared as well.
They also want to share the data
with the entirely different companies or the enterprise
who they called as a partners
and as well as their customers.
Now, let's see,
what is AWS Organization and AWS Lake Formation
AWS Organization helps you centrally manage
and count your environment
as you grow and scale your AWS resources.
Using AWS Organization,
you can programmatically create new AWS accounts
and allocate resources,
group accounts to organize your workflows,
apply policy to those accounts
or the groups for those governance,
and simplify billing
by using a simple and a single payment method
for all your accounts.
In addition,
AWS Organization is integrated with AWS Services,
so you can define central configuration,
security mechanism, audit requirements, and resource sharing
across accounts in organization as well.
AWS Lake Formation is fully managed service
that makes it easy to build a secure and manage data lake.
Lake Formation simplifies and automate
many of the complex manual steps
that are usually required to create data lakes.
And these steps include collecting, cleansing,
moving, cataloging the data,
and securely making the data available
for analytics and machine learnings.
Lake Formation provides its own permission model
that augments the IAM permission model.
This centrally defined permission model
enables fine grain access to the data stored
in data lake through a simple grant and rework mechanism,
which is very similar to relational database.
Lake Formation enables federated data lakes
with cross account sharing,
delivering centralized domain-oriented data products
across your organization,
using well gone data sharing
with minimal to no data movements.
Now, let's say you have account A
with the databases and tables,
and you want to share with the account B and C directly,
you can do that as of now.
But you want to share
with both of the account A and account C in a single grant,
then organization units of organization come into picture.
You can share at the organization unit level as well,
and not only organization unit,
you can share with entire organization
where all the accounts under that organization,
the resources will be get shared.
And this is what you're going to see in today's demo.
So Lake Formation uses AWS Resource Access Manager service
to share resources.
If the grantee account is
in the same organization as the granter account,
the shared resources is available immediately to grantee.
So last thing, if an account is not part of organization,
you can still be able to share externally as well.
Let's get into demo now.
For demo, we are going to use AWS Management Console,
and currently logged in as Administrator.
We gonna quickly check the AWS Organization.
For AWS Organization,
we have a Master Account who is going to
manage all the accounts.
So we call it as a Management Account.
And we have grouped into organization's unit.
Organization's units here is Marketing and Sales,
and quickly add an account into this
by sending an invitation.
For that,
we are going to do through Invite.
You can place the account number here,
send invite,
and then I'll be receiving the invitation.
I accepted the invitation from the target account,
and if I refresh this page,
I'll be seeing the new account being shown
in the organization.
I'll move this to Sales.
Okay. So now, we can see all the accounts being added
and then organized into different organization unit.
Now, in order to use data sharing with AWS Organization,
we had to go to Resource Access Manager Console Page
and then go to Settings.
In the Settings,
check box Enable Data Sharing with AWS Organizations.
By doing this way,
you'll allow Lake Formation to share your resources
using the AWS organization.
Next, we will go into Producer Account.
So currently what we have done here is we have
this Management Account for organization,
which will be acting as a Producer Account
which will create data on S3 and catalog it using Glue.
And that we'll be using
as a resource to share with the entire organization.
So for that, I'll quickly change the profile to
Lake Formation Administrator,
who is going to share this data
between different accounts using organization.
Go to AWS Lake Formation,
and this will be the Lake Formation console.
Here I have created a database,
and which has certain tables registered under S3 location.
So before we start granting for these resources,
we need to register the S3 bucket
under our Data Lake Location.
So we'll click on Data Lake Location,
and Register Location,
and then go by AWS S3 path,
and then Register Location.
Next, what you need to do here is go to database
and start granting access to these resources.
We'll quickly check what is the permissions
of this database.
So we see that LakeFormationPrimaryAdmin
is the database administrator for this,
and next is we'll grant,
click on Grants and then External Accounts.
And then once you click on the search bar,
you'll start listing all the options to share.
So you will see the AWS Organizations unit
and as well as Sales.
For this demo,
we'll share this database or the tables to the Organization.
All the accounts under Organization
will get these resources.
Click on that,
and account name, Resources,
choose the database, database name is test-sharing-ou-db,
and will give Describe permissions,
and Grantable Permission as Describe as well.
A quick review on this and then click on Grant.
And then we go to database,
see the permissions.
We see organizations has been added.
Next thing is will give grant permissions to databases
and as well as tables.
So next click on Grant, and then select External Account,
and then Organization Unit,
and we'll give permissions
to all the tables that needs to be shared.
And will give permissions as Select and Describe,
and click Grant.
Now we will go into consumer account.
So for consumer account,
I have currently logged in as a AWS administrator.
And for this demo,
we are be using Lake Formation administrator
who is going to share the data,
what we got from producer account
to the resources within this account.
And that resources can be something
like account analyst who has Athena privileges
to query the data.
So for this,
again, we had to go to Lake Formation,
so I'll quickly change the role
to Lake Formation Administrator,
and go to Lake Formation.
Once you are in the Lake Formation console,
you'll see the shared database that has came
from your producer account.
You consume this by creating a resource link
on this database.
To create resource link,
click on the Database and then click on Create Database.
And here, there is the option of Resource Link.
We'll get a look friendly name,
and we'll select the shared database,
and then Create.
Once you do this,
you'll see the permission for this database
by going into Permission section.
So in this, you'll see, IAMAllowedPrincipals
are being added.
So we are going to rework this permission,
and give only Lake Formation secondary administrator
who has a privilege to grant.
Before giving the grants,
we'll check in the Athena
to see whether it has all the permission.
So I see the database and has all the tables.
Now what I'll do is I'll switch the profile to Analyst
and see whether he's able to see this database and tables.
For him,
this database as well as stables are not being granted yet.
So we had to go back
to Lake Formation and grant access to him.
Again, I'll switch back to Lake Formation Admin.
Next thing I'll do is click on the database,
the local database we created,
and Grant.
And I'll choose the administrator role as Analyst role,
and Describe privileges,
and Grant.
So next thing you're going to do is
we'll do the same thing by doing the Grant on target.
We can go to database,
verify the permissions.
So we see the secondary analyst has the permission now.
Now we'll go to Athena,
and switch the profile to Analyst.
Now you see the database has been reflected
and all the tables.
You can preview the data by clicking on three dots,
and then Preview the table.
And then you can see the data has been reflected here.
So business analyst has access
to the data that has been shared from using Organization.
So this concludes the demo where Lake Formation
with the integration with the organization
will help you seamless data sharing.
Thank you.