- [Swagat] Hello, and welcome back to the second episode
of the video series focusing on Amazon Comprehend,
a fully managed
and continuously trained natural language processing service
that uses machine learning to uncover valuable insights
and connections in text.
I am Swagat Kulkarni, Senior Solutions Architect in AWS,
and today, I'll be talking about Amazon Comprehend
pre-trained models and APIs.
It all starts with your data.
The data can consist of variety of documents, or text,
such as social interactions, logs, emails,
phone conversations, and many more.
Next, you input the data to Amazon Comprehend,
and use pre-trained models to extract insights
from the data.
These models are exposed by way of easy to consume APIs.
There are multiple ways of interacting with these APIs
starting with AWS Command Line Interface, or CLI,
programmatically by using SDKs
in multiple programming languages, such as Java and Python,
or AWS Console.
Let's take a look at these APIs in action
from the AWS Console first,
followed by programmatic access using Python.
After you login to AWS Console,
type comprehend in the search bar,
and then click on the result that shows Amazon Comprehend.
Next, click on Launch Amazon Comprehend.
This takes you to a page where you can perform
real-time analysis on the input text.
As you can see, there are two analysis type,
Built-in and Custom,
and today, we'll focus on the Built-in analysis type
which leverages the pre-trained models.
You will notice that a sample text is
already available for analysis.
Let's click the Analyze button now.
Once the analysis is completed,
Amazon Comprehend presents the insights gained
from the input text.
Let's start with the Entities tab first.
An entity is textual reference to the unique name
of a real-world object, such as people, places,
and commercial items.
Comprehend can recognize up to nine different entities
in the input documents.
Here, you can see Amazon Comprehend API call returned
the pre-trained entities and its Type from the text
along with the Confidence score.
The Confidence score is a number between zero and one.
Low Confidence score means that the service's confidence
in its output is low.
Conversely, if the service is highly confident of its output
the score will be closer to one.
Next, we'll look at Key phrases tab.
A key phrase is a string containing a noun phrase
and the modifier, such as adjectives
that describes a particular thing.
Here, you can see the list of noun phrases
and its modifier that Amazon Comprehend detected,
and the associated Confidence level.
The next tab is Language.
This tab shows the dominant language of text
and Amazon Comprehend's level of confidence
that it has detected the dominant language correctly.
In this case, it correctly identified English
as the dominant language with a high confidence score
of 0.99.
Amazon Comprehend can now recognize 100 different languages.
The PII tab lists entities in your input text
that contain Personally Identifiable Information, or PII.
A PII entity is a textual reference to personal data
that could be used to identify an individual.
Examples include an address, bank account number,
or phone number.
There are two different modes here.
The Offset analysis mode identifies the location
of PII in your text documents.
The Labels analysis mode checks for the presence of PII
in your text document and returns the labels
of identified PII entity types.
As you can see in the Offset mode,
Amazon Comprehend returned the actual values
of the PII entity types, such as name, credit card number,
bank account number, et cetera.
When we toggle to the Labels mode,
the API return just the entity Type,
such as name, email, bank account number,
and corresponding Confidence score.
The Sentiment tab shows the dominant sentiment of the text.
Sentiment can be rated as neutral, positive,
negative, or mixed.
In this scenario, you can see
Amazon Comprehend detected the sentiment as Neutral
with a high confidence score of 0.98.
And finally, the Syntax tab shows a breakdown
of each element in the text along with its Part of speech,
such as proper noun, verb, pronoun, et cetera,
and the associated Confidence score.
You can use the results of the syntactical analysis
to gain a richer understanding of the content
of your documents.
Next, let's explore ways to programmatically access
the pre-trained APIs using the Python SDK.
Here we are showing a Jupyter Notebook,
which provides step-by-step instructions
to use Amazon Comprehend APIs to uncover information
in unstructured data.
We will explore six pre-trained APIs
identifying named entities, extracting key phrases,
identifying the dominant language,
determining emotional sentiment, determining syntax,
and detecting Personally Identifiable Information, or PII.
Let's look at the cells in this notebook one by one,
and their outputs.
We start by importing Boto3.
Boto3 is the name of the Python SDK for AWS.
Boto3 makes it easy to integrate your Python applications,
library, or script with AWS services.
We will use this SDK to work with Amazon Comprehend
pre-trained APIs in this notebook.
Next, we specify the region
where this notebook instance is running in,
import other packages, such as JSON, Pandas, and NumPy,
and then use Boto3 SDK to create a Comprehend client object.
Next, we'll look at the input text.
Here, you will see the same sample_text
that was used in the Amazon Comprehend console.
First up, we'll start with identifying named entities.
Named entities can be real-world object
denoted with a proper name.
In this cell, we have invoked DetectEntities API
by the sample_text and the LanguageCode of the text,
in this case, English.
The response is a JSON output containing Entities array.
Each object in the array has the location of the entity
in the text identified by the Begin and EndOffset value,
the confidence Score of the model,
the actual Text, and the Type of entity,
such as person, organization, quantity, et cetera.
The output of the next cell shows the same output
in a more human readable format, as shown here.
Next, we use the same text to detect key phrases.
Here, we have invoked DetectKeyPhrases API
by passing the sample_text and the LanguageCode.
The API response is an array of KeyPhrases,
and each object contains the Begin and EndOffsets,
the confidence Scores, and the actual key phrase.
Let's keep cruising, and next up is language detection.
Here, we have invoked DetectDominantLanguage API,
and passed the sample_text.
If more than one language is detected,
the API returns detected language
and its corresponding score.
The response is an array of languages, as shown here.
The API has correctly identified English
as the dominant language
with a high confidence Score of 99%.
Again, we show the same output in a human readable format.
Next is sentiment analysis,
and the API used is called DetectSentiment.
As you can see by now, the API input follows
the familiar pattern of passing the sample_text
along with the LanguageCode.
The response returns the Sentiment value.
In this case, the value is NEUTRAL,
along with the SentimentScore,
which is similar to the confidence score,
and has values ranging between zero and one
assigned to each sentiment.
You can see the Neutral sentiment has the highest score here
of approximately 0.98.
Here is the reformatted output
showing the Sentiment value
and the corresponding sentiment Score.
Next is syntax detection.
It is used to identify the part of speech
for each word in the text.
Here, we have invoked DetectSyntax API.
The API response returns an array of syntax tokens
with each member element of the array consisting
of Offsets, PartOfSpeech Score and Tag
to denote the value, such as interjection,
proper noun, verb, et cetera,
along with the Text and the TokenId.
Again, you can see the readable version of the same output
with the Text, Part of Speech, and corresponding Score.
Finally, we look
at Personally Identifiable Information detection.
Here, we have invoked DetectPiiEntities API
to detect the PII information in the input text,
such as name, address, bank account number.
Following the familiar pattern,
the API response returns the Offset of the PII value,
the confidence Score, and the corresponding PII Type.
Here is the same output in a readable format.
We can also use ContainsPiiEntities API
to label documents that contain PII.
As you can see in the output,
the API response returned five labels identified
from the text and the confidence Score for each one of them.
And here is the reformatted version of the same output.
This brings us to the end of this notebook.
In this notebook, we learned how to use
Amazon Comprehend pre-trained APIs for various scenarios
and invoke them using AWS Python SDK.
Thank you very much for watching this video.
I hope you enjoyed learning
about Amazon Comprehend pre-trained models and APIs.
Go ahead and check out the resources shown here
to explore further.
Please keep an eye for upcoming videos on Amazon Comprehend
and how you can use it to solve business problems.