We may have run a plethora of NLP and ML
operations in various data processing pipelines.
Yet, what should we do should we wish to make them scalable and want to automate
the administration of the pre-trained models?
Stick around.
Dataflow ML, one of the most recent Google Cloud offerings, is here for our rescue.
Hi, this is Aniket Agrawal, working for Google as a Strategic Cloud Engineer.
Our primary objective is to perform text clustering within its Dataflow pipeline.
Here, we'll rely on Apache Beam JupyterLab extension to execute the entire pipeline.
At high level, we'll use Dataflow ML
to administer the pre-trained models for spaCy and scikit-learn sequentially.
The spaCy model will be used to vectorize
the words and convert them into their 300 dimensional word embeddings.
Then, these vectors will be assigned
to their respective scikit-learn BIRCH clusters.
Then, the pipeline data or the
PCollection output can be visualized easily using the 'Interactive Beam' tab,
which is differentiator in this particular notebook.
In today's world,
NLP is becoming exceedingly important as more and more businesses and organisations
seek to derive insights from the text data.
Hence, there are a lot of points which do
really justify the importance of word clustering.
It can help us certainly in enhancing the search engine results,
in simplifying document text analytics, and quicker retrieval of significant
information from various documents and large data sets.
Let's dive into a few theoretical particulars.
So without further ado, let's get started.
First, we'll examine what Dataflow is, why it's so important, and how it works.
At a high level, Dataflow is a fully-managed service
for creating, building, and executing the Apache Beam pipelines.
It achieves this in a cost effective,
fast, and serverless manner within the Google Cloud ecosystem.
Chunks called 'PCollections' can be
processed easily using functions called 'PTransforms'.
Dataflow can handle both batch and stream
processing and can automatically scale to meet demand.
Therefore, it provides a powerful tool
for analyzing data in real-time, for analytics, and various ETL operations.
This is the central theme of this entire
video for which we were desperately waiting, right?
We'll be exploring the key benefits
and features of Dataflow ML for running ML pipelines at scale.
At its core, Dataflow ML allows you
to run ML models on large datasets using Apache Beam.
To do so, it combines the capabilities
of Dataflow with Apache Beam's 'RunInference' API.
Hence, with Dataflow ML, you can easily process massive data sets,
train complex ML models, and deploy them to production with ease.
There are diverse handling patterns
for the models, notably, 'sequential' and 'branching'.
As per the below architecture diagram,
we'll use spaCy and scikit-learn model handlers, sequentially.
These vectors are clustered using scikit-learn
BIRCH clustering, after which the final findings are reported and visualized.
Now, we'll take a closer look at what spaCy is and why it's such an important
and powerful tool when it comes to natural language processing (NLP).
SpaCy is an open source Python NLP
library that is designed to be fast, efficient, and easy to use.
It provides a wide range of tools
and functionalities for performing various NLP tasks, including tokenization,
Part-of-Speech (POS) tagging, and Named Entity Recognition (NER).
To achieve all of this,
it provides various pre-trained models that can be used out-of-the-box in order
to generate vectors or embeddings for the text.
For clustering the word vectors, scikit-learn BIRCH clustering is employed.
It is a hierarchical algorithm that is designed to be fast and memory-efficient,
making it very tailored for the large data sets.
And finally, BIRCH Clustering is easy
to use with scikit-learn to cluster your text in any kind of data set.
Enough of theoretical details!
Now, let's head straight to the notebook demo.
First, we'll create an Apache Beam Vertex AI Workbench notebook.
For that, visit the 'Dataflow' section in the GCP console.
Click on 'Workbench' followed by 'User Managed Notebooks'.
Now, to create a new notebook,
click on the '+' sign followed by 'Apache Beam' and 'Without GPUs'.
After specifying name, region, network, click on 'Create'.
I'll move ahead with the default parameters.
Wait for2-3 minutes for the 'Open JupyterLab' option to become enabled.
Click on 'Open JupyterLab'.
Once the launcher opens up, click on 'Apache Beam 2.46.0 for Python 3'.
You can begin executing all of the mentioned code cells here onwards.
Now the actual implementation fun begins.
First things first.
We'll install all the required packages, including spaCy and scikit-learn.
As the kernel selected is 'Apache Beam'
rather than 'Python', the installation instructions will differ slightly.
Enable all the relevant APIs such as Cloud Storage, Dataflow, and Notebook.
It can be done in code or via the GCP console.
Here, we'll focus on the programmatic approach.
Using 'gcloud' command, we'll list all the services and take
into account the titles corresponding to the relevant service names.
Now, we'll enable Cloud Storage, Dataflow, and Notebook APIs.
Now all you need are training and testing
data sets containing a lot of relevant words.
The sample files are supplied for your
convenience in testing in the description below.
In totality, 307 words belong into four categories, namely 'wildlife', 'food',
'medical', and 'technology' are taken into account.
If we read the test and train data
files, we can easily observe that 196 words are
taken for testing purpose and 111 are meant for the training.
Now, we'll run a simple text vectorization code cell.
For this, we download and load the spaCy model.
Allow at least 1-2 minutes for the process to complete.
We'll apply it on our example word 'Google'.
For creating this vector,
the loaded model 'nlp' essentially passes text through several phases like
tokenization, Part-of-speech tagging, and Named Entity Recognition.
We run this code cell and there you go, a 300 dimensional vector is produced.
For all of the 111 words of the training dataset, 111 word vectors are created
using the large model loaded in the previous step.
This results in a 2D matrix having 111 rows and 300 columns.
With the help of this matrix,
the model is trained and this is the output indicating four clusters.
The prediction results indicate cluster
0 assignments followed by 3s followed by 2s and finally by 1s.
So, if you look at the entire CSV file, you can easily observe
that the cluster 0, in fact, corresponds to the 'wildlife' related terms,
for example, 'animals', 'habitat', 'biodiversity'.
This is followed by cluster 3,
which is meant for 'food' related terms such as 'chocolate', 'fruits' and 'vegetables'.
This is followed by terms such as
'computers', 'smartphone', 'tablets' assigned to cluster 2 for 'technology'.
And finally, the last set of words belong
to Cluster 1 for 'medical', including 'epidemic', 'fever', 'flu', etc.
Now, we'll work with Beam's RunInference API for Sequential Model Handling.
For this, we need to define and create the handlers.
First, we'll create the spaCy model
handler for vectorizing words for which necessary modules are imported first.
The class 'SpacyInfer' is created to leverage spaCy for inference.
It will define, load, and apply the model.
The main logic is encapsulated inside
the function 'run_inference', which creates 300 dimensional embeddings
for each and every single word and stores them in a list called 'infers'.
Similarly, for clustering vectors,
scikit-learn model handler is created for handling Numpy arrays.
Again, we begin with the necessary imports.
We'll dump the BIRCH Clustering model trained earlier into a pickle file
which is further passed as a model URI input for the handler.
Now comes the main step.
We'll build and execute the entire word clustering pipeline.
Here, we'll rely on Interactive runner
for iterative development and inspection of pipelines.
So after specifying the runner as
interactive, we define and run the pipeline.
First, we'll extract all the words
from the CSV file, then spaCy RunInference operation is
performed on these words to create their 300 dimensional embeddings.
This is followed by executing scikit-learn
RunInference to predict clusters for these 300 dimensional vectors.
As you can see in the output, Prediction Result is consisting of two components.
One, the array which represents the 300 dimensional embedding for the word,
and the cluster 0 is inferred for this particular vector.
And the model ID is the URI which we
actually talked about earlier, that is in the form of pickle file.
The entire procedure is repeated for all the 196 test words,
and we'll see this output in the form of pandas dataframe.
The word to cluster assignment could be
visualized using Apache Beam Interactive Beam Module.
As you can see, the first set of words
assigned to the 'wildlife' cluster, i.e. Cluster 0,
followed by the next set of 'food' related terms assigned Cluster 3.
They are 0 false positives and false negatives for this small data set.
As the dataset grows larger, 100% accuracy cannot be guaranteed,
and then the medical terms are assigned Cluster 1.
There are a few challenges that do come into picture.
For example, the word 'virus' can belong
to the 'technical' as well as the 'wildlife' cluster.
Similarly, the word 'meat' can also be related with 'wildlife' as well as 'food'.
If a word is supposed to be placed in more
than one cluster, it is referred to as 'fuzzy' clustering.
Our focus for this demo will be 'hard'
clustering where each word will belong to exactly one cluster.
And finally, the 'tech' set involving the word 'Google' is assigned to Cluster 2.
We'll visualize the pipeline data or the output in the form of PCollection.
This could be done programmatically using 'Apache Beam Interactive Beam' modules,
but our focus will be on using the 'Interactive Beam' tab,
which is specifically provided by this particular notebook.
That's all for part one,
but don't go anywhere, as still there is a lot more to come.
In the next part, we'll visualize all of these
PCollections and see the entire pipeline getting exported as a Dataflow job.
So, see you over there.
Thanks a lot for viewing this one :)
Let's sign off for now !!