in this video I'm gonna show you how to sync local files to azure blob storage
using AZcopy
AZcopy version 10 became available in late April of 2019 the
latest version has an option that piqued my interest the delete destination
parameter used with the sync option will sync a source folder to blob storage and
remove files from the destination if they're no longer on the source before
that real quick please take a second to subscribe and lick the bell icon to get
notifications of new content click sorry typo
click the bell icon the sync and delete destination option gives az copy a
feature comparable to robocopy with a mirror parameter providing the ability
to keep the blob container in sync with the local file system or other blob
containers for that matter this is a great option for ad-hoc backups or any
time keeping an up-to-date cloud copy of a file structure is needed this gets
even better when combined with Azure blob storage soft delete a soft delete
will allow you to recover a deleted blob for any given number of days using
azure sync with soft deletes has the potential to be a compelling backup
option ok enough of me yapping let's get into the demo buckle up everyone
it's demo time I'm gonna start out in my azure storage account and I'm gonna
create a blob container this is gonna be the destination target for the sync
operation and I'll give this the name Cir AZ
copy sync test and I'll leave it private click OK
and now we have it created so I'm gonna go in go to properties and copy this URL
I'm just gonna paste that right into here and I really don't care that you're
seeing it because this will be long gone by the time you watch this video the
next thing I'm going to do is come back to
my blob and I'm gonna enable soft delete so let's go down here and what soft delete
does is it allows you to recover files deleted and here you can see it gives me
seven days to recover those this isn't like a snapshot this is just think of it
as a wastebasket this is a precaution in case you need to recover anything delete
it by mistake so I'm gonna save that and the next thing I'm gonna do is come to
shared access signatures there it is I'll leave this with the defaults for
now and I'll just generate a SAS connection string and what I need here
is the SAS token so I'm gonna copy that then I'm gonna come back to notepad
paste that in so that gives me the URL and the SAS token all in one string I'll
need that for the AZ copy command now let's go over to my local file you can
see I have a few of them open here here's the test data that I'm gonna copy
but you know what before I do that I need to download and install the AZ
command so here is microsoft's website and it has three different options for
the AZ copy Windows Linux and mac so you can get the file from here I already
have it downloaded and I'm gonna go over to here's the download directory and
it's downloaded compressed so I have that unzipped on the Windows machine I'm
just gonna take this azcopy.exe I'll copy it I'm gonna bring it over
here to C:\windows\system32 and I'm just gonna drop it in here and it will
prompt me for my password okay so now I have the AZ copy command in the system32
directory that's part of the path variable and Windows so I should have no
problem running AZ copy from the command prompt and we can give that a try
oops
there we go that shows us all the commands available I'm just gonna clear
that out okay so now I'm gonna go back to my source you can see that I have a
test data folder here under a Z copy test and in it I've just got a bunch of
random files different sizes different names but simple text files that should
be sufficient for testing so I'm going to come over here that's going to be my
source so let's start building this command so I'm going to start with AZ
copy and I'll specify sync then I'll drop the path in so that's the source
first and then I'm the destination and remember I have that right here so I'm
going to just copy this closes it with a quote and there's a couple options I can
put at the end one is reclusive that will transverse sub-directories that's
on by default so I'm gonna leave that off the other one though is a delete
destination since we're doing an initial sync there's really nothing to delete at
the destination but I'm going to add it in here anyway
so I've found 35 files and it's uploading them okay to view these now
I'm gonna use storage Explorer Here I am in storage Explorer and you can see I've
got my storage account and my blob container and if we go in there we can
see my test data folder and if I go in there we have 35 items so that did
exactly what we wanted it to do it's the initial sync now we've let's pay
attention to 30 through 35 let's remove those and see what happens so I'm going
to go back to the source and we'll just remove 30 to 35 so they're gone from the
source and I'm going to run this command again so now you can see it deleted
those five files which is good so let's go back into storage Explorer we'll do a
refresh and yep sure enough 30 35 are gone okay let's go back to the storage
account in the portal and we'll go back to blobs and then go into the blob
container here we can see test data showing there and here all the files and
I one thing I wanted to show you in here is if I click this show deleted blobs we
should be able to see there they are so that's because of the soft delete is
enabled that I could go in and manage these as well okay so that works pretty
well let's go in and create a subfolder because I want to make sure that the
recursive file sync is actually working without that option so we go to folder
we'll call this test one
and I'll just copy all these right into there we'll run this again there goes it
is uploading those files now one thing I should point out I ran into an
interesting issue with us the virtual machine I'm running this on had a single
core and when I was testing this the AZ command would bring the processor to a
hundred percent and just hang and what I found is that there is a bug in the
version 10 that will exhibit that behavior on a single core machine so I
gave the virtual machine a second core which it probably should have had in the
first place and that issue went away so in case you do run into that azcopy
at least at this time seems to require multiple cores in order to run
successfully ok so there's some great detail on what it did and let's go back
to file explorer and do a refresh and there's the test okay so those files are
located in that subfolder so that is syncing recursively now let's go back
and try something different we're gonna go back to the source and I'll go into
that newly created test folder and I'm gonna delete out a lot of them this time
I'm gonna run the sync operation without that delete destination command there
goes so you can see it didn't actually delete anything now if I come back here
do a refresh you can see all the files are still there they weren't deleted
from the destination although they do not exist at the source so let's run
that command again this time putting the delete destination back there goes and a
simple refresh again should show us yep it got rid of all those so that's pretty
cool that now with AZ copy you can keep blob storage container in sync with a
local filesystem or vice versa or between blob storage accounts I even saw
a note that you can use AZ copy now to copy data from AWS s3 bucket into blob
storage so that's it for the demo that how to use AZcopy to keep blob storage
in complete sync with a local file system that is it I hope you found this
useful don't forget to subscribe and click the bell icon for notifications of
new videos thanks for watching