ANUBHAV SHANKAR: In the previous video,
we talked about how to use the Compute Engine set of Google
Cloud activities within the saw vean management using
the instance activities that is already present here.
The second part of this particular tutorial
is going to be around these storage activities
and how we can use them to create and manipulate buckets
and same goes for the objects as well.
So as a use case, right now what I have in store
is that we want to do some kind of object
migration, some kind of data migration
from Azure to Google Cloud.
And for that, we have used the Azure activities as well
to actually read the data.
And we are going to use the Google Cloud
activities to actually create the corresponding buckets
and the files that is going to be
stored within that container.
So let me explain to you what is already in place for you
and what we need to create as part
of this particular tutorial.
So first of all reading from the Azure as already done for you,
we have an Azure scope activity that is
going to connect to the Azure.
It is going to get the storage account based
on the name and resource group.
It is then going to iterate over all the containers
that is present within this particular storage account,
basically it is also going to go to the level of blobs.
And in here once based on each container
we are going to create corresponding buckets
and Google Cloud.
That is one of the items that we are going to do.
And the second we are going to iterate
for each of the blob that is stored within the container,
we are going to download the blob into a file,
and then we are going to transfer
that file into the newly created bucket in the Google Cloud.
So that's the total flow of this particular project.
And then what we need to complete
is a creation of the bucket in Google Cloud.
And then we are also going to design transferring of the file
into the Google Cloud bucket that we have created
while reading the issue.
So let me show you what we have in place from the Azure
perspective.
So this is the strategic goal that we
are going to use within that.
I already have a container in place.
I'm going to expose that this container is named as storage
and it contains two files-- one as PDF, one as test image.
And this is what we have to transfer to Google Cloud
buckets.
So in here, let me go to the buckets.
So you can see that there is only one bucket already present
and probably it is going to create a corresponding bucket
for the newly transferred data storage that we have in Azure.
So for that, let's start building our project.
So first of all, as part of this we
are going to create corresponding buckets
to the Azure.
And in here, I have all the items
that I need to create the project.
I have the service account key file,
I have the project activity.
And I want to create this bucket and I
will get the bucket name and bucket name basically
will be corresponding to one that is coming from the Azure
naming and the bucket will help us in getting this bucket out
of this particular file.
So let's create this one.
This should be a pretty straightforward.
So first of all, what I'm going to do
is obviously use a Google Cloud scope to attach to the instance
that I'm going to work with, and in here I'm
going to use service key file that I have.
So this will attach to the instance.
And then I'm going to actually create a bucket naming.
Like I said, there are certain naming requirements that
is there that you need to follow, and, in this sample,
because the name of the container that we had in Azure
was storage and storage is one of the names that is basically
restricted to use while creating the names of the buckets,
that is what I'm going to do.
I'm going to replace the storage part
with an abbreviated version of it
so that it gets accepted while I create it.
And then I'm going to create tags for the ownership.
In this particular one, I am going to have only one tag
called creator so that this resource belongs to me.
And I'm going to store it in labels.
And then I'm going to create the bucket.
And for creating the bucket, I have the activity already here.
I can drag it and drop it on the workflow.
And in here, I have to provide certain items.
First is the bucket name, second is the project ID
where I want to create this.
Then I have to choose the region.
This could be depended upon what kind of performance
and what kind of availability you want.
So you can select all these three or only one
based on your configuration.
I'm going to use only one that I've been using so far.
For the output of the bucket, I'm
going to latch it to the out argument
that I have in terms of out bucket,
and I think that should be it.
Yes, I have to attach the labels as well.
So I'll put in labels in here.
Right.
So you have other options to choose from.
For example, storage class, you can have it
on archive mode, coldline mode, or standard or nearby.
I'll keep it as standard right now.
And you have also option of choosing from GoogleManageKey
or if you have a CustomerMangageKey
you can choose that as well.
So all those options you get it out of the box
while using these activities in here as well.
So this is ready.
I can run this particular low profile,
but I'll do that once I complete both the part of it.
So once the bucket is created and I have downloaded the file,
I want to upload that file to the Google Cloud.
And for that I have this workflow
that we need to develop.
And again, I have all the arguments in place
that I would need to create this one.
So to begin with, again, I'm going to use Google Cloud
and then I'm going to use service key file.
I have this service key file here.
This will do.
And then what I need to do is to actually upload the file.
And for uploading the file, I have an activity
that is created, which is called upload object from file.
And in here, I have all the information
that I need to provide.
For example the bucket.
This is the bucket that I created in the previous file.
Content type is going to come from the information
that we downloaded from the Azure.
Same goes for file path.
And I can keep the object name as I please.
So in here, I can provide the object name same
as the file name.
I can provide a timeout for running this activity as it
will run until completion.
And then I can provide the storage class.
And thus again, I can choose from the option that I have.
I'll keep it standard for now.
And that's pretty much it.
That should do.
So I have created both of these low profile.
And now if I run this, I should be
able to transfer my complete set of files
that is there in my Azure instance
into the Google Cloud platform.
So it is creating the bucket now.
It has created a bucket.
Now, it is going to upload the files.
It has uploaded the first file.
It is going to do it again for the second file.
It has completed uploading it to the second file,
and that's pretty much it.
It's done.
So if I go back to the instance, if I refresh this,
I will get the new Azure storage in here.
This is the one that is created right now.
If I go inside this, I should have both those files available
with me.
I have the sample file and I have the test image file.
These were the files that we had within our storage Azure,
sample file and the test file.
And this is exactly what has been transferred here.
So that's pretty much it.
For this particular video, you saw
how to use the storage of a set of activities
within Google Cloud and how you can
use it to migrate data from Azure instance
to Google Cloud instance.
Thank you for watching.
Reboot Work.