(upbeat music)
- Welcome back to another episode of Azure AI Essentials.
Today, we're going to take a look
at how you can scale your applications
with Azure Cognitive Services.
Because this is such a critical topic
with a lot of rich information, I've invited Amy,
one of Azure AI's expert cloud advocates
to help us unpack this topic.
- Hi Merna, it's great to be chatting with you today.
And I'm really excited to be talking
about Azure Cognitive Services, how they scale themselves
but also how they can help you scale your projects.
- Great. Let's get started with the basics.
We often talk about scalability
when it comes to cloud services,
and it's a very popular reason
for moving applications to the cloud.
So what is scalability
and why is it so important to consider early on
in app development?
- Great question.
Scalability is the ability to grow your business
or your service on a sort of pay as you go format.
Meaning, there's no sort of high upfront investments
on machines, et cetera.
There's also no need to have specific types of technology
for specific solutions or trying to put things
where they shouldn't be going,
but actually, you get full flexibility
to create that solution.
There's also some cool stuff now with cloud computing
where scaling features are there to help you automatically.
So you might look at things like Scaling Vertically,
which means scaling up the size of a machine.
Or you might look at Scaling Horizontally,
where you're scaling basically the number
of concurrent occurrences of those machines
in order to use that workload well.
- So since I'm working with AI developer technology,
specifically Azure Cognitive Services,
what do I need to consider with regards to scaling?
Do I need to consider both Horizontal
and Vertical Scaling features?
- Well, actually, Azure Cognitive Services
has really been built with scalability in mind.
And so one of the key things I thought about
was the ability for, instead of us having to think
about scaling or out of machines for Cognitive Services,
it kinda does it all for you, which is great.
One of the best ways that I kind of think about this
in my mind is serverless compute.
So serverless computer has become very, very popular
and it's the idea of running code
and not having to worry about the infrastructure
that it runs on.
Well, Cognitive Services is very, very similar.
So if I have an Azure function and I call it 10 times
or 1000 times, I still write exactly the same code
for those two different transactions.
It's the same with Cognitive Services.
If I call the computer vision API
and ask it to describe 10 images
or ask it to describe 1000 images, it's exactly the same.
And so the service has actually already dealt
with the scalability for you.
- Wow.
So I don't really have to do anything.
Azure Cognitive Services takes care of it all.
- Exactly.
The next way,
actually that Azure Cognitive Services has been built
with scale in mind, is actually considering
sort of the changes in your API usage.
So say your application goes viral,
you see a huge influx of usage of this feature.
So within the Cognitive Services,
you can actually scale up
say the front end of your application
and maybe the backend of your application
and Azure can help you with that as well.
But actually the API, the Azure Cognitive Services API
will scale automatically for you
so you don't need to worry too much about that.
- So developers don't need to worry
about scaling this part of the solution architecture either.
It's no wonder we see fascinating statistics
across our services.
Just in March, 2021 alone,
we saw 53 billion transactions
using Cognitive Services APIs.
- Exactly right.
Huge numbers of many, many people
are scaling their applications.
And actually, that's where I kind of wanted
our last consideration to go.
Azure AI can help you scale your projects
and the pace at which you create those projects.
So if we think about these AI models that Microsoft builds,
they have incredibly large data sets
and they spend time training
and building these different models
for you to use through an API call.
If you think about the opposite of that,
if you were to build it yourself,
you'd need extremely large and rich data sets.
If you were in a translation scenario,
you'd actually need lots of different language data sets.
And you'd also need the infrastructure
to be able to train these.
And so when you start breaking it down,
you can see actually
just being able to call an API very quickly
and completely translate some of your documents,
it is incredibly amazing.
- This is great.
There's so many ways these services help scale
when building applications
and supporting different business processes.
Do you have an example that illustrates
some of these ideas in action?
- Yeah, so one story that really illustrates
some of the points we've just talked about
is actually the work done.
So Vodafone is a global telecommunications company
and they're actively expanding their digital strategy.
They need flexible, scalable technology solutions
to meet these digital ambitions.
Vodafone use Cognitive Services
to develop a personal digital assistant.
And they named it TOBi.
TOBis work in their customer service interactions
and helping support all those interactions.
They built this bot using the Microsoft Bot Framework,
the Language Understanding Service,
as well as many of the other Azure Cognitive Services.
One of the really nice pieces was how easily
they could connect all their technologies
within their business as well.
They had really positive results early on
that the system was working incredibly well
in a customer service interaction.
Vodafone wanted to scale the solution much further.
And so they first deployed TOBi into Italy
but decided they actually wanted language specific versions
in 15 other markets.
So in order to adopt a bot in a country's local language,
Vodafone adopted the translator
which is a Azure Cognitive Services
that allows for real time AI based text translation.
The company currently makes TOBi available
in 15 different languages and they have even more plans.
So scaling from one to 15 different languages
was massively accelerated by the use of these APIs.
And I mean, the general scale also speaks for itself.
Vodafone reported that in December, 2020,
TOBi now holds 25 to 30 million conversations a month
with its customers.
And it handles 60% of all customer interactions.
- So many conversations with TOBi.
This is a use case that truly demonstrates the scalability
of our AI services.
And in this case, it was essential
that our AI solutions were easy to use
for Vodafone's conversational designers
to build modular technologies
to support the company's ambitions
and make TOBi the biggest chat bot in the world.
Okay, so now I'm curious, can you show us the technology
that Vodafone used for translation?
- Yeah, of course.
So let me show you the translator service in action.
Most sites, websites, documents, et cetera,
will have to go through a lengthy
and expensive translation process.
Any time any new texts or changes are made to the document,
with the translation service,
you can save your files, be that HTML, PDF, DOCX, et cetera,
and translate them into multiple different languages.
So up to 90 different languages and dialects.
- But what if I have a graph in the middle of my document,
what language would it show up in?
- That's a great question.
Check out this complex document that was translated
using the Translator API.
So in this case,
we can see the whole document's been translated
and it kind of looks like that was the original language
it was meant to be in.
So in the case of adding a graph here,
that would also be translated
and show as if it was the language that had been created in.
Something else I wanted to mention that might help
with these types of things, these complex documents
is also adding translation systems
that are built using the Custom Translator.
The Custom Translator's available
through the same cloud APIs that are highly scalable,
highly secure, et cetera.
But the Custom Translator
actually provides neural machine translation
for these custom models.
This results in really great quality translation
and also the understanding
of your specific business terminology.
In order to do this, you can actually provide
your own previously translated documents
to build up this customized translation system.
So it will really add that extra quality
if you need it with specific terminology.
- It's interesting you give that example.
As I know we use our APIs at scale
within many of our own products internally as well.
For example, Metrics Advisor.
Another one of Azure Cognitive Services uses AI
to perform data monitoring and anomaly detection
in time series data.
- Absolutely.
These APIs by design are used for enterprise
and production workloads.
And that's why they're built
into our first party services for you to use.
- Given that most areas of scalability
are taken care of for developers
when it comes to Azure Cognitive Services,
are there any other considerations they should keep in mind?
- There are a couple of design points to consider
when it comes to using Cognitive Services.
For example, if you're thinking
about using them outside of the cloud.
So one of the reasons they're all nanny
that you might want to call the Cognitive Services
outside the cloud
are is if you need high throughput
and low latency as a requirement.
So you need to be able to send many transactions
as quickly as possible.
In this case,
you could deploy the Azure Cognitive Services
physically closer to that application logic and your data.
And it would then for-sit
in a container that you're able to use and still use
the amazing API that's available from Azure.
Containers don't cap transactions per second
and can be made to actually scale up and out
just like we spoke about at the start,
as long as you provide the necessary hardware resources
for that.
- That's great to know.
Is there anything else developers should keep in mind
when scaling with Azure Cognitive Services?
- Yeah, another consideration can be made
when scaling Cognitive Services, applications,
to big data workloads.
So Azure Cognitive Services for big data,
let's use this term, terabytes of data,
through our Cognitive Services
using the popular big data framework called Apache Spark.
With only a few lines of code,
Azure Cognitive Services can be used
with Apache Spark API
in the Microsoft machine learning spark Namespace.
This also applies to Scala and Java
which are incredibly popular with big data languages.
- This has been great.
Thanks so much for joining us, Amy,
and for explaining all things scale
when it comes to Azure Cognitive Services.
- Of course, thanks so much
for inviting me onto Azure AI Essentials.
- We've covered so much great information today.
Azure Cognitive Services has truly been built
with scalability in mind.
And it's easy to see that
just by looking at how simple it is
to scale Cognitive Services for large workloads.
Whether your customer's usage
of Cognitive Services increases or decreases,
and regardless of your innovation pace,
Azure AI supports your innovation scalability
so you can focus on the rest of development.
We're excited to see what you build
using Azure AI solutions,
knowing that your applications can seamlessly scale.
Thanks for joining us on another episode
of Azure AI Essentials.
Until next time.
(upbeat music)