data is one of those things that makes the world go round
everything uses data just think about it for a second from the device in your
pocket social media the store down the street down to the large enterprise and
governments everyone has data and that means that everyone needs to store that
data someplace so for the next few minutes I want to talk to you about all
the recent updates to storage in Azure
I'm Dean Cefola and this is the Azure Academy Azure storage is a fully managed
platform service in the cloud that is highly scalable in size and performance
can be backed up and protected from disasters with the press of a button
it's accessible anywhere in the world and you can build it in just seconds and
there are many different offerings for Azure storage so let's look at the
documentation and in case you don't know the quick link to get here is a kms /a
sure docks and when you get here we'll scroll down and go to the storage
section on the left and here are all of the different offerings around Azure
storage and these break down into a few different families we have the storage
that's related to high performance compute and that's the fxt storage and
HPC cache then we have storage related to data operations like data Lake and as
your data shares and then we have these storage account related items like
archive storage as ER files blob storage queue and table storage then we have the
on-prem services like data box and store simple so there's a lot of different
ways that you can interact with storage in the cloud so how you do it really
depends on your workload we're gonna start off with blob storage here and
let's go over to the azure portal I've got a resource group called blob
replication and in here I've got two storage accounts in the Canada East
region and that's where these preview features are enabled so if I go into the
first storage account two containers I've got a container called test and
then another one here that's called dollar blob change feed and change feed
is one of the preview features and there's two other features that are also
being used here and that is indexing and object replication so if I go into the
test container you see I've got two different files and if we look back at
the other storage account I've also got a container here called test where I
have the two same files so how am i able to do this well if I go to the storage
account and I scroll down to the blob services section we have a new feature
flag that you can enable called object replication and here's where you can set
up replication rules - asynchronously copy data from one
container to another across multiple storage accounts and across multiple
Azure regions now in the docs under the blob storage section in the filter you
can just type object replication and then click the first link and this tells
you how object replication works and that it is something that is
asynchronous and on the right here we've got a link for about the preview telling
you what regions you can enable this in and that there are other things that go
along with object replication like blob versioning change feed and then you can
also use another feature called indexing and this makes your storage account much
more powerful rather than just being able to contain your data it actually
can help you to manage your data and then down here in the register for the
preview section you've got the powershell to directly run and set up
your storage account and so all i did was i copied this code and then ran this
in my powershell and i waited for all of the providers to be registered and then
i was able to go in and create the storage accounts and use this feature so
you can see I've already got an object replication rule here replicating from
blob rep zero zero zero over to this account which is zero zero one and I can
click over here on the three dots to edit the rule so starting at the top you
can see we've got our source storage account which is 0 0 0 destination is 0
0 1 and then the particular rule for the container my source and destination were
both called tests but they didn't have to be they could have been called
anything and I'm not using any filters so let me show you how this works we're
gonna go and create a new container and I'll call this as your Academy and hit
create and inside my new container I'll click upload and upload some files and
here's some JSON documents and we'll click open and then upload those and
then I have to go to the other storage account and create a container here to
receive those items and I can call this whatever I want and I'll just call it
replicated and hit create alright so let's go back to the source storage
account and we'll scroll down in the left here to blob services and object
replication and notice that I've deleted my replication rule so I could show you
the whole process at the top we click setup replication and we have to enter
our death the nation details starting with our
subscription and then our storage account then we select our source
container in this case as your Academy and our destination which is replicated
then if there are any filters that we want we can add them here as well and
I'll enter the word azure here as I had one file that started with Azure all the
rest of them start with the word desktop and then you can use the copy over
feature to determine how you want to copy things either copy everything in
the account or only new objects from here on or you can have your own custom
replication settings I'm gonna just choose to copy everything and we'll hit
save now remember everything in this case is everything that matches our
filter okay so I'm not moving every file over just the ones that match and then
you can write other things here if you want as well like I could put back my
test folder and notice the copy over here is only new objects so I've got to
upload a new object into my source account before it will replicate to my
destination then I'll just hit save and apply and there are our two rules
so I'll click under these source containers here my test folder since I
need to upload a new file before it's going to replicate and I'll click the
upload go to select a new file and then I'll just copy in some PowerShell files
and upload those and that completed successfully so if I go back to the
object replication I can navigate from here to my destination container and
just be aware that it may not happen immediately because it is an
asynchronous process and there's all of our PowerShell files now if I go to my
replicated folder we can see in there that it has finished the copy of my JSON
file because it began with the word adder so it met my filter condition and
replicated the file over back in the replication rules I can click to edit
the rule set and I'll change my filter here and I'll delete the word asier so
that it copies over everything in the account I'll hit save and apply and if I
go back to the replicated folder still hasn't come over yet might take another
minute oh there it is everything is now replicated so this very easily allows
you to have control over replicating data in your blob storage accounts
however it is that you want to manage from account to account or from
container to container even across multiple regions so I'm gonna go back to
my Azure Academy source container and I'm gonna delete a few files here the
ones that are the desktop workspace contributor there we go those are gone
so now my replication of my deletes should also carry over let's flip back
to our destination container that's called replicated and those have deleted
now well if we go back to our source storage account I want to turn on
another feature here and it's down in the blob services area under data
protection and that would be the soft delete feature now soft delete not only
will protect us from files that are deleted but also files that are
overwritten and here's also where you can enable the versioning and change fee
and those are requirements for replication which would have happened
when you did the initial PowerShell so with this enabled we can go back to our
tainer so if i click this button up here to
show our deleted blobs there's our three that were deleted and if I want it to
restore one of these files I could just click on the three dots and go to view
the previous versions find the particular version of the file that I
want let's say this one and click to make it the current version and that'll
bring it back from the delete of course another thing that we can do is
snapshots and we can create a snapshot of any of the files that are here which
I have done and then you can click on the individual file and view any of the
snapshots and depending on the file type you can't even go into edit here and you
can manipulate the file but that's gonna be specific to the kind of file that it
is there's also the concept of axis tiers and so I can check on this
snapshot and go to change the tier and I can go from hot to cool to archive and
doing that can dramatically reduce the cost of retaining the data but
understand that there is a performance penalty the colder you get so let's
transition here from blob storage into files and to do that we're going to talk
about a feature that is for both of these services and that is called
private link private link allows you to set up a IP address on your private as
your virtual network that is representative of that service
so in the case of storage you would have the IP address of 10.1.1.10 --ts as your
storage in your account we then disabled the public internet
endpoint and the only way to communicate with it is over your private endpoint so
back in our storage account in the main part of the blade we'll scroll down a
little bit to the Settings section and you can see private endpoint connections
is here and at the top you can click to add a private endpoint and Wendy do you
first choose your subscription and then your resource group just like all other
Azure resources and I'll call this instance private blobs and it's in
Canada East because that's where my storage account is and we'll click Next
for my resources and in the resource types this is all of the platform
services so here we need to find storage and notice that there are two options
there's storage accounts and then there is these storage sync service and this
is a j''r files sync and if you're unfamiliar with that service go and
check out our video on as your files sync on the top here the great way to
bring your on-prem files server information into the cloud and turn your
file servers into hot file caches but here we'll just choose our storage
accounts and then for our resource will pick the individual storage account that
we want and because storage accounts have many different kinds of resources
we can even isolate down to blobs in this case and we'll click next for our
configuration and we've got our network already set up and I've got a subnet
selected here for this as well and at the bottom we have our private DNS
integration and this is gonna create for us an azure private DNS resource and
it's going to register our storage accounts blob service as that resource
with an IP address and we'll hit next and then add our tags and everything
looks like we're good to go so let's create while that's going on
we'll go to our other storage account and we'll go to add a new file share and
we'll click the file share at the top and we can give it a name and then we
have the quota and this is how large of a file share we want to create and if I
try to create a share that is 10 terabytes in size we've got a problem
and that is that we can only create one that's up to five tera
bytes in size now this is the default behavior in our storage counts but there
is a way to fix this so we'll discard those changes and in
the settings section of the blade we'll go to configuration and here's where you
can change all of the behaviors around the storage account we have a feature
called large file shares and this is the one that's going to help us solve the
five terabyte issue by enabling large file shares we can go as high as 100
terabytes though I'll hit save for that and let's go back to our file share now
and create that new share and now when we put in our 10 terabytes no problem
how about we go to a hundred no problem let me go bigger than 100 nope 100 is
the maximum alright so I'll just create a hundred terabyte file share and there
we go created just in one second so now that
that's created let me go through and set up the private link for that and where
we go to select our sub target resource we'll be selecting files here and we'll
click Next for our configuration and we want this to be on our files subnet and
we will again set up a private DNS zone and then add our tags and then review
and create and now that that's complete we have our two private endpoints each
one of them corresponds to the network card and that network card is what gives
them the IP addresses on our private virtual network and then we have our two
DNS zones so first let's take a look at our network there's the two network
cards and there are IP addresses for blobs and files and if we go to one of
the DNS zones in the center here we've got our SOA record and also the IP
address from that network card that we just created which is registered in this
zone and then over on the left we can see that it is linked over to our
virtual network so that enables you to talk to this storage account privately
but we're not yet complete because we still have the internet facing endpoint
that we need to disable and for that we go into the storage account and on the
left here we go to the firewall and virtual network and you see by default
that this is unable inform all networks and that does include the internet so we
want to turn this to selected networks and now the question is which
worx need access to this storage account we need to add those Azure virtual
networks up top as well as anything else from on-prem that we need to gain access
so this could be your front end NetID public IP address from on-prem or the IP
for your virtual network gateways connection on Prem or an individual
client IP so I'll check the box here to add my client IP and I'll click the plus
to add my existing virtual network select the correct subscription and then
the virtual network itself then select which particular subnet I want to allow
in this case the blob and then we'll click enable and now that that has been
enabled we'll click Add to complete the process
you click Save and let's go and repeat that on our other storage account and
we'll click Add for that as well then at the end of all that you click
Save and that'll change the configuration on the storage account and
with that complete the only way to access the blob or files from these
storage accounts is over our private IP address we're going back to the storage
account with our file share in it to other things that have been improved
we now have soft delete enabled to protect our file shares and this is a
storage account level process so we'll open up my file share here and I'll just
upload all these files ok and that is complete now in a file share you can
also change the size of the share itself either up or down I'm going to go here
from having 100 terabytes down to just having one terabyte and again that
process just happens instantly and file shares have multiple ways of being
protected as well so under snapshots we can just take a snapshot here of our
share and that is of the entire file share as is there you can see all of the
individual files so if we go back to the overview and I was to delete a couple
and now I need to retrieve them so I could go to my snapshot and then go and
retrieve them from here and so I'll take this work bat file and just click on the
3 dots and hit restore and when you do you can choose to overwrite the existing
file or file name if it was present or you could give it a new one and I'll
call this restored one dot bat it okay and if we go back to the file
share itself there is our restored file that protects the files inside the share
what happens if the whole file share gets destroyed oops it's gone what do I
do now so I've seen this show deleted shares button so if I click that up nope
nothing here how do I set this up so in the main blade I've got to scroll down
to the file service section and here's where a soft delete is located so I've
got to enable this here and then set my retention which is seven days by default
now hit save for that this is again a setting for the entire storage account
I'll create a new share all this share to create that one terabyte share again
they'll upload those files now I'll just go and delete the share again and now I
can go and hit the show deleted shares so if we click on the three dots and do
the undelete it is now restored and you can click on the share and there's all
of our data so we can protect the individual files in the share with a
snapshot we can certainly protect it without your backup but now soft delete
also enables us to protect the entire file share so another way to manage and
control your share one benefit that I didn't mention yet in the file shares is
that the individual file sizes that you could upload has now increased to four
terabytes that means now you can store things like VHD files on your file
shares think about things like FS logics and storing things like app attached in
the near future and with the individual file size of 4 terabytes now we should
be able to for the foreseeable future accommodate any applications or profile
issues and other file data that you need and speaking of use cases like FS logics
if we scroll back up in the blade to our configuration of course we have done a
video already on using AD authentication with your storage accounts so go check
that out in the top if you had missed that and this feature has also recently
been made generally available today you can get your file shares for FS logics
Windows virtual desktop or any other use case involving an azure file share
authenticated to your Active Directory that can all be secured with a private
endpoint and keep your user profile safe so I hope you've enjoyed looking at this
storage roundup all of the new features that have
recently become available in preview or generally available for use in storage
accounts all different ways that Microsoft is enhancing the management of
the platform so that you can do what you do best and if you like this video or
learn something new today go ahead and click that thumbs up and let me know
that you appreciated our content and please feel free to share this video
with others who would benefit from it as well as subscribe if you haven't done
that already and join us here at our community where we're just trying to
help everybody learn all they can about Azure and if you have any questions
about today's video or any comments on anything around Azure or something that
you'd like to see in a future video give me that in the comments field down below
and we'll get to work on that right away and if you're looking for something new
to watch over here on the right we've got our latest video as well as one
we've selected just for you thanks for joining us for today's video and we will
see you soon happy learning