Welcome, everyone, to this session,
during the Storage track of Azure IaaS Day.
I am Karl Rautenstrauch.
I'm a Principal Program Manager with the Azure Storage team,
here to talk to you today about migrating
your NAS data to Azure,
and I am broadcasting from my home office
and pandemic safe house, here in North Florida.
So, thank you for joining me
and for, hopefully, enjoying my decor here at home.
-: We're going to start off with a quick view
into our Azure Storage portfolio.
So I like to say that no matter what your,
or your application needs are from a storage platform,
we can meet those needs,
and we can do that through block storage,
in the form of our Azure disk portfolio,
object storage for those of you who need REST API access
or a data lake in the cloud,
and then finally, our file storage solutions,
Azure Files and Azure NetApp Files,
which give you SMB and NFS access
for those traditional applications
that need shared data platforms.
And we're going to focus, during today's session,
on the final two.
On our object storage and on our file storage platforms
and most importantly, how you can begin using
them knowing that you have file storage in other locations.
And really file storage is generated by applications
and users.
So no matter what your use case is for adopting services
in the Azure public cloud,
you will need to bring files to support those.
Whether you're moving an analytics platform,
beginning the path towards AI and ML based applications,
moving your virtual desktop infrastructure to Azure,
or just your server-based applications,
I can guarantee that you having data on NAS platforms,
or many of you in on-premises object storage platforms
that will need to come with those applications,
and the how-to-move-that-data is critically important,
and that's what we're going to focus on here today.
So, why can't you just use
embedded operating system tools
or tools that we make available
for every single file migration use case?
Well, there are several reasons
why they may not be the best solution.
Whether it's rsync in the Linux and Unix space,
RoboCopy, or our own AzCopy or Azure Data Box platforms,
they're not always the best thing to meet your needs.
I saw this myself as an IT pro.
I had to move tens of terabytes
to hundreds of terabytes of data at a time,
and using professional tailor-made tools
made my life significantly easier,
especially when you're moving between disparate platforms.
When you're not moving from one generation to another
of a vendor's hardware,
it can be really, really difficult to copy files en masse.
So what these solutions bring to bear is security.
They encrypt that data in transit.
They give you fantastic monitoring of the activities,
so you know how much longer you have to go,
how far into the process you are.
When errors occur, you're notified and you're given
the opportunity to resolve them very quickly,
and they're all built to move data quickly.
They provide scale-out copy engines that you can deploy near
your data to make sure it can move across
the wire as quickly as possible.
And they're automated.
Configure them, set them to run,
and then you don't need to intervene until it's time
to perform that cut-over
from your old on-premise or cloud environment to Azure.
So, so many reasons to use this software,
to bring it into your process, and make your life easier.
But I do want to be clear
that they're not necessary
for every file migration to Azure,
and I'm going to start with why you may not want
to use solutions like this, or not need to.
And the first reason is the data type.
So if you're looking to move data that resides
on tapes today,
these solutions cannot address that data.
The solutions I'm going to spend the rest of
the time talking about.
Instead, you can use your existing backup software
to recall from tape and write that to Azure.
But what if you're not using backup software?
If you have some other solution you've used
to write that data to tape?
We have a great partnership
with a company called TapeArk
that can help you extract that data from tape,
and load it into cloud storage.
If you have less than 50 terabytes to move,
it may not make sense to use these solutions.
They do require setup,
but they do require a little bit of a learning curve.
So investing that time may not be worth it
for less than 50 terabytes,
but I don't want to make that a hard rule.
One that is a hard rule is for databases
and virtual machines that are stored on NAS.
These are not the right solutions to move
that rapidly changing, individual file level data
from one NAS platform to another.
It's better to use database replication software,
or something like VMware's own storage vMotion
to move from NAS on premises
to NAS in Azure for those storage use cases.
Where they do make sense is almost
the opposite of everything that I just described.
When you have more than 50 terabytes,
you absolutely need a solution like this
to help make that process a smooth one.
If you are doing this as part of a larger migration project,
it's great to have orchestration across the board,
and to be able to tie these things together and again,
make your life as simple as possible.
And finally, when you do have the bandwidth
to migrate over the wire, these are great solutions
that can help you fill that pipe.
They also have solutions that make sure they don't fill
that pipe when you don't need them to,
and some of them can write to Azure data box.
So if you want to use a combination
of offline ingest and online ingest,
they can help you meet those needs
and use the best combination of scenarios
for your particular environment.
So let's take a look at what a file migration workflow
should consist of when it's executed in a safe
and proper manner.
It starts with step one A,
and that is understanding where in Azure
you should be moving your data to.
We've taken a lot of time over the last couple of years
to compile resources,
to help our customers understand the SMB protocol options,
the NFS protocol options,
and our object and data lake storage options
that are available to customers in Azure.
So please take the time to review these assets and resources
to understand what is the right platform and solution
for you.
You also want to take a look at the tools
that are available.
We do have partnerships
with several strategic partners in this space.
So let's call this step one B,
and this is evaluating the tools themselves,
comparing them to see which one best meets
your source environment, destination environment,
and individual copy parameter needs.
We have an exhaustive review of these solutions available
for you.
We've also worked with these partners to build videos,
showcasing exactly how their technology works,
and we've also built getting started guides with them
that provide you with the steps you need to deploy
and begin using them for your file copy needs.
Now, the next step in the process
is to actually deploy these solutions.
And they are available in the Azure marketplace.
So you can find them there by searching for file migration.
Each one of these providers will have a contact me option,
so you can learn more from them about their solution,
and perhaps get a training session to really hit
the ground running, once you actually deploy
the copy engines and the management console,
and those copy engines will need to be deployed on premises.
So you'll have to have available server infrastructure
to deploy them to,
and the vast, vast majority of customers deploy them
on top of their VMware or Hyper-V infrastructure.
They really don't need a dedicated physical server
in order to function properly.
The management console, that's up to you.
You can deploy that on premises.
You can deploy that in Azure.
Please do note that anything you deploy in Azure,
from a virtual machine perspective, is a billable service.
You will have to pay for that VM while it's running,
providing that management console or the copy engine,
depending on your scenario.
Maybe the copy engines will reside in Azure as well,
but keep that in mind through the deployment process.
Now the next step is to actually begin the migration itself.
So this step is comprised of choosing my source system
where my data resides currently,
my destination system,
where I want to copy that file data to,
setting my copy parameters, like what to do
if I'm copying, for example,
data that belongs to an orphaned NTFS SID.
What should I do with that?
Who should become the new owner?
And then kicking off the migration process.
And this is a background activity that will copy
while users are still leveraging
the on-premise or cloud-based current production system,
and copying in the background.
You have the ability to monitor that copy in the background.
You know, I mentioned one of the greatest virtues
that these solutions bring to bear
is the fact that you have an estimate
of how much longer this copy activity
is going to take, any percent complete status.
It's a little bit different from what we're all used
to with Windows File Explorer.
And then the final phase, step four,
is to perform the actual cut-over of your shares,
or exports to the new environment in Azure.
And I do have to recommend to everyone
that you do some testing before you perform that cut-over.
One, you should make sure that you never get rid
of the original copy.
That's just a belt and suspenders safety model
that an old CIS admin recommends to everyone else.
Make sure things have completed properly
before getting rid of the original copy.
So mount up those shares and exports,
make sure that the security parameters,
the ACLs translated over properly,
and then move on to that cut-over phase.
And that cut-over phase is something that
you will be scheduling during a change window.
That is going to require coordination
with the application owners and the end users.
So again, old CIS admin offering some advice there.
And then finally, when the cut-over is complete,
you're going to migrate these shares or exports.
on the new target virtual desktops or virtual servers
in Azure.
Here, in this example,
I'm mounting a Azure NetApp Files volume
that I have migrated content to
from an on-premise NetApp system.
So I'm a big believer in showing you the look and feel
and experience.
We've walked through the process,
but let's take a look at a demo and see what this looks like
in action.
Okay.
Now we're going to take a look at one of the solutions
that you saw during the presentation.
We're going to look at Komprise
and their intelligent data management suite.
Now here, they have really powerful capabilities
that allow you to see things like potential cost savings
by tiering some of your data to object storage,
versus copying it all between different file platforms.
So they have powerful assessment
and analyzation capabilities.
They also support, obviously, migrating files
between disparate sources,
replicating files between disparate sources,
if you want to use this solution for DR
and lifecycle management,
to make sure that you can keep the right data on the right
cost and performance tier over time.
Really, really powerful suite here.
And we're going to look at now where it all begins,
and that is adding in the existing file environments
that you have,
the ones that you're considering migrating to Azure.
So you can see Komprise has a really robust list
of arrays that they support,
and we're going to quick go down the path
of adding a Clustered Data ONTAP system.
I can either add the entire cluster
and look at the resources there,
or just a specific storage virtual machine
and restrict my view to that point,
and see, I've got quite a few different sources
added here already.
Now we're going to take a look at targets,
and targets are object storage targets for tiering.
Well, let's go ahead and add a Microsoft Azure
blob storage account, or start down that process anyway,
where I would need the name,
the destination container within that account,
the storage account access key,
and to choose the default tier that this data would live on.
And this is a capability that I really like,
which is to keep the files in their native format.
So I can then make them available to Azure services,
like Azure ML.
You can also chunk and compress them to make the storage
a little more efficient,
if you want to use this as an archive.
Now we're going to take a look at the cost modeling
and the analysis.
So these values are ones that you populate,
based on your pricing from Azure.
You can compare that to the pricing
for your on-premise environment,
and you know exactly what your savings will be really
for you, for the dollars that you pay for this.
And then you can also model all of the migration activity
and throttle the migration activity,
based on a percentage of bandwidth that you have,
you want to make available to these tiering
and migration activities,
and, you know, I love this view as we look across
the types of files that we have,
the size of those files,
who in our user base is consuming that space,
which departments in that user space is consuming
all of my disk capacity.
And then what system is most full?
Where should I start with migrations?
All of these things are available to you after you run
the initial assessment analytics after adding the shares.
So now that we know what we have,
let's go ahead and begin a migration.
And that migration all starts by adding in a source
and target pairing.
We'll take a look at that here in a second.
Okay.
So here, I'm going to choose my source.
In this case,
I'm going to migrate from an on-premises SMB file server
I have.
We're going to choose a target.
Azure Files will work,
and then I see those warnings there.
Things I need to consider.
I can go ahead and read more about that,
and once I get my source and destination,
I need to provide a friendly name
so I know exactly what this is when I look back
at that table containing all of my migration activities.
And I get to set a couple of settings here.
Of course, very, very important
to preserve the ACLs.
I absolutely want to do that.
Keep that one on.
And then I start my migration,
and it will pop up here along with
the two preexisting ones that I had.
I can go ahead and choose from a variety of activities,
including pausing it,
if, let's say, we have network congestion,
I can pause and resume later.
Start file iteration.
That is when I'm getting ready to perform the cut-over.
we're going to take a look at some of the details here,
the progress of these jobs.
There's some incredible amount of detail
for every one of the activities.
You know, of course, you can refresh at any time
and see how far along it is, how things are moving.
Let's go ahead and look at the details for
this particular job.
And there we go.
I can see how much data has been moved in each one
of these iterations after the baseline,
get an idea of what my change rate is,
how many files,
how much data your changes on a regular basis,
and then when I'm ready to go during that controlled window,
I can go ahead and start my cut-over and we're done.
All right.
Great to share that demo with you.
I love putting my hands on keyboard and mouse
and actually using these solutions.
Another great part of my job in working with partners.
I do want to highlight a few last things before
we part ways today.
One is, if you're like me,
you don't always necessarily know who is using a file share
or an NFS export.
I ran into this as an IT admin a couple of years ago.
I had a mainframe application that was connecting
to one of my NFS exports that I had no idea existed,
or was using my environment as the Unix admin in that way.
So fortunately, we provide a tool that can help.
Azure Migrate has a dependency mapper capability
that allows you to run and look
for open ports, ports in use on servers
in your environment.
So you can find those SMB protocol
and NFS protocol connections
that you may not know had existed in servers in
your environment.
So a great way to avoid
that call to the tech support desk down the road,
after you performed that cut-over.
And really if you need help during your migration,
there are many resources that can help.
I encourage you to go and take a look at
the Azure Migration and Modernization Program.
Not only do we have SIs that can help,
but we have Microsoft experts as well as part of our AMP
program that can help you with not just your file migration,
but server and database migration,
and really any considerations
that you have in your journey to Azure cloud.
And then once again,
we've built a lot of resources over the last year,
year and a half, to help you determine
what is the best SMB protocol, NFS protocol, object,
or data lake storage solution for your needs,
and even educate you on what object
and data lake storage are,
if those are things you're unfamiliar with.
So please take advantage of the URLs
that I've embedded here.
Spend a few minutes and read up on these solutions,
and you'll be better for it
as you're planning your journey to Public Cloud.
So finally, I'd like to say thank you for your time today.
I've really enjoyed delivering this session,
recording these demos.
Can't wait to interact with you all going forward
on the tech community and in person going forward,
as we get back to in-person events.
Thank you very much for your time.
Take care.