Hello Everyone Today we will talk about Azure Blob Storage on IoT Edge
What is it?
It is a light weight Azure-consistent module which provides block blob storage.
It comes with configurable ability to automatically upload the data to Azure and provides support for intermittent internet connectivity.
It also has the ability to automatically delete the data from the local blob storage account
What all we will learn today?
How to configure and deploy this module?
deviceToCloudUpload functionality in Action
deviceAutoDelete functionality in Action
deviceToCloudUpload functionality in a disconnected scenario
How we will be doing it?
We already have an Azure IoT Edge device running Azure IoT Edge Runtime.
We will deploy Azure Blob Storage on IoT Edge module, pointing to edge local blob storage account.
We will install Azure Storage Explorer in a desktop, and connect it to our local storage Account and Azure storage account
Then we will upload data to our local blob storage account via Azure storage Explorer and check out deviceToCloudUpload in action.
Sounds good? Let's get started
Number-1 - How to configure and deploy this module
For this demo, we will be using VSCode
We have an Azure IoT device running Azure IoT runtime.
We added this section
let's call our module localstorage
Provide the link to the mcr URL
Provide a name to your local storage account
Provide local storage account key
And Storage Path. If you are in Linux use volume mount in the format shown in the call out
Now, let's configure deviceToCloudUploadProperties and deviceAutoDeleteProperties as desired properties for this module
For deviceToCloudUploadProperties
Set uploadOn to true to turn ON this feature
The uploadOrder is set to OldestFirst, it means my oldest data will be uploaded first.
cloudStorageConnectionString expects connection string to your Azure storage account where the data will be uploaded
storageContainersforUpload property allows you to specify the source and target container names.
In this example, we are saying upload the data of cont1 in local blob storage to edge-cont1 in Azure Blob Storage
For cont2 am using template strings to specify my target container name
For cont3 you can see I have not provided the target container name, hence the module will automatically assign it one.
deleteAfterUpload property when set to true will automatically delete the data from local storage account after it uploads the data
For deviceAutoDeleteProperties
Set deleteOn to true to turn ON this feature
deleteAfterMinutes is set to 5 mins, it means the module will delete the blob from local blob storage account after 5 mins of its last modified time.
retainWhileUploading when set to true will retain the blob while it is uploading even if deleteAfterMinutes value expires
Let's deploy this module
Now, let's refresh. Here is our module.
Let's make sure I can ping my iot device
I have downloaded and installed the Azure Storage Explorer
Now let's connect it to our local Storage Account.
I'll show the connection string format on the screen.
Lets create container Cont1 in local storage account.
Similarly, I created cont2, cont3 and cont4.
Number 2- Now its time to see deviceToCloudUpload functionality in Action
I have connected my Azure Storage Account as well, let's make sure I don't have any pre-existing containers.
Now, let's add an image to cont1 in the local blob storage account
Wait for a while and refresh.
The image magically appears in our Azure storage account.
Let's make sure it’s the same image I uploaded
Let's see what happened when we added some files to cont2 and cont3
the files appeared in the container in Azure whose name are assigned appropriately as intended
Cont4 is not listed under storageContainersForUpload
hence any data added in this container will not be uploaded to Azure
Number 3- Lets see deviceAutoDelete functionality in Action
But before that lets discuss couple of data delete behaviors
which you can achieve by using deviceToCloudUploadProperties and deviceAutoDeleteProperties in combination
Here are the values you can specify to delete the data from local storage account as soon as "deleteAfterMinutes" value expires
With these values in place lets upload a blob, the module will try its best to upload the blob before the "deleteAfterMinutes" value expires
But it will not retain the blob in local storage account if it cannot.
You will find that The blob is automatically deleted from the local storage account, once their deleteAfterMinutes value expires
Here are the values you can specify to delete the data only, when data is successfully uploaded.
I have set deleteAfterMinutes to 1 minute and "retainwhileTiering" & "deleteAfterUpload" are set to true.
I added an 11.7 GB blob to the local storage account. It takes more than 1 minute to upload to Azure.
You will see even though "deleteAfterMinutes" value has expired it did not delete the blob from the local storage account
Once the module ensures that it successfully uploaded the blob it will automatically delete the blob from the local storage account.
Number 4- Let's see deviceToCloudUpload in a disconnected scenario.
Am adding a blob of roughly 11.7 GB to our local storage account.
After it is added to the local storage account
let's wait for a while and disconnect the internet from the iot device.
You can see the files did not appear in the Azure storage account.
It means it didnot finish uploading
Let's wait for a while and reconnect the internet.
The module automatically figured out that the internet connection is back again and started uploading the blob.
And Here is my blob in Azure
For more up-to-date information go to these URLs.
Thanks for watching