In this course you will learn all about natural language processing and how to apply it to real
world problems using the spacey library. Dr. Mattingly is extremely knowledgeable in this area,
and he's an excellent teacher. Hi, and welcome to this video. My name is Dr. William Mattingly,
and I specialize in multilingual natural language processing, I come to NLP from a
humanities perspective, I have my PhD in medieval history, but I use spacey on a regular basis to
do all of my NLP needs. So what you're going to get out of this video over the next few hours
is a basic understanding of what natural language processing is or NLP, and also how to apply it
to domain specific problems, or problems that exist within your own area of expertise. I happen
to use this all the time to analyze historical documents, or financial documents for my own
personal investments. Over the next few hours, you're going to learn a lot about NLP language as
a whole and most importantly, the spacey library. I like the spacey library because it's easy to
use, and easy to also implement really kind of general solutions to general problems with the
off the shelf models that are already available to you. I'm going to walk you through in part one
of this video series how to get the most out of spacey with these off the shelf features.
In part two, we're going to start tackling some of the features that don't exist in off the shelf
models. And I'm going to show you how to use rules based pipes or components in spacey to actually
sole domain specific problems and your own area from the entity ruler to the matcher to actually
injecting robust complex regular expression or regex patterns, and a custom spacey component
that doesn't actually exist at the moment. I'm going to be showing you all that in part two,
so that in part three, we can take the lessons that we learned in part one and part two, and
actually apply them to solve a very kind of common problem that exists in NLP and that is information
extraction from financial documents. So finding things that are of relevance, such as stocks,
markets, indexes and stock exchanges. If you join me over the next few hours, you will leave this
lesson with a good understanding of the standing of spacey and also a good understanding of kind
of the off the shelf components that are there and a way to take the off the shelf components
and apply them to your own domain. If you also join me in this video and you like it, please let
me know in the comments down below because I am interested in making a second part to this video
that will explore not only the rules based aspects of spacey, but the machine learning based aspects
of spacey. So teaching you how to train your own models to do your own things such as
training a dependency parser, training a named entity recognizer things like this, which are not
covered in this video. Nevertheless, if you join me for this one and you like it, you will find
part two, much easier to understand. So sit back, relax, and let's jump into what NLP is, what kind
of things you can do with NLP such as information extraction, and what the spacey library is and how
this course will be laid out. If you like this video, also consider subscribing to my channel
Python tutorials for digital humanities, which is linked in the description down below.
Even if you're not a digital humanists like me, you will find these Python tutorials useful
because they take Python and make it accessible to students of all levels. specifically those who
are beginners, I walk you through not only the basics of Python, but also I walk you through
step by step some of the more common libraries that you need. A lot of the channel deals with
texts or text based problems. But other content deals with things like machine learning, and
image classification and OCR, all in Python. So before we begin with spacey, I think we should
spend a little bit of time talking about what NLP or natural language processing actually is.
Natural Language Processing is the process by which we try to get a computer system
to understand and parse and extract human language oftentimes with raw text. There are a couple
different areas of natural language processing. There's named entity recognition, part of speech
tagging, syntactic parsing, text categorization, also known as text classification, co reference
resolution machine translation. Adjacent to NLP is another kind of computational linguistics field
called natural language understanding NLU This is where we train computer systems to
do things like relation extraction, semantic parsing, question and answering this is where
bots really kind of come into play, summarization, sentiment analysis and paraphrasing. NLP and NLU
are used by a wide array of industries, from finance industry, all the way through to law
and academia with researchers trying to do information extraction from texts. Within an LP,
there's a couple different applications. The first and probably the most important is information
extraction. This is the process by which we try to get a computer system to extract information that
we find relevant to our own research or needs. So for example, as we're gonna see, in part three of
this video, when we need to apply spacey to the financial sector, a person interested in finances
might need an LP to go through and extract things like company names, stocks, indexes, things that
are referenced within maybe news articles, from Reuters to New York Times to Wall Street Journal.
This is an example of using NLP to extract information. A good way to think about NLP
is application in this area, is it takes in some unstructured data, in this case, raw text, and
extracts structured data from it or metadata. So it finds the things that you want it to find and
extracts them for you. Now while there's ways to do this with gazetteers, and list matching, using
an NLP framework, like spacey, which I'll talk about in just a second, has certain advantages,
the main one being that you can use and leverage things that have been parsed syntactically or
semantically. So things like the part of speech of a word things like its dependencies, things like
its co reference, these are things that the spacey framework allow for you to do off the shelf,
and also train into machine learning models, and work into pipelines with rules. So that's kind of
one aspect of NLP. And one way it's used. Another way it's used is to read in data and classify it.
This is known as text categorization. And we see that on the left hand side of this image,
text categorization or text classification. And we conclude in this sentiment analysis for the most
part as well, is a way we take information into a computer system, again, unstructured data or
raw text, and we classify it in some way. you've actually seen this at work for many decades now,
with spam detection, spam detection is nearly perfect, it needs to be continually updated.
But for the most part, it is a solved problem. The reason why you have emails that automatically go
to your spam folder, is because there's a machine learning model that sits on the background of
your on the back end of your email server. And what it does is it actually looks at the emails,
it sees if it fat fits the pattern for what it's seen as spam before, and it assigns it a spam
label. This is known as classification. This is also used by researchers, especially in the legal
industry, lawyers oftentimes receive hundreds of 1000s of documents, if not millions of documents,
they don't necessarily have the human time to go through and analyze every single document
verbatim. It is important to kind of get a quick umbrella sense of the documents without actually
having to go through and read them page by page. And so what lawyers will oftentimes do
is use NLP to do classification and information extraction, they will find keywords that are
relevant to their case, or they will find documents that are classified according to the
relevant fields of their case. And that way, they can take a million documents and reduce it down
to maybe only a handful, maybe 1000 that they have to read verbatim. This is a real world application
of NLP or natural language processing. And both of these tasks can be achieved through
the spacey framework. spacey is a framework for doing NLP right now. As of 2021, it's only
available I believe in Python, I think there is a community that's working on an application with R
but I don't know that for certain. But spacey is one of many NLP frameworks that Python has
available. If you're interested in looking at all of them, you can explore things like NLT Kay, the
natural language toolkit stanza, which I believe is coming out of the same program at Stanford.
There's many out there, but I find spacey to be the best of all of them for a couple different
reasons. Reason one is that they provide for you off the shelf models that benchmark very well
meaning they perform very quickly. And they also have very good accuracy metrics such as precision
recall, and F score. And I'm not going to talk too much about the way we measure machine learning
accuracy right now, but know that they are quite good. Second, spacey has the ability to leverage
current natural language processing methods, specifically transformer models, also known,
usually kind of collectively as Bert models, even though that's not entirely accurate, but it allows
for you to use an off the shelf transformer model. And third, it provides the framework for
doing custom training relatively easily compared to these other NLP frameworks that are out there.
Finally, the fourth reason why I picked spacey over other NLP frameworks is because it scales
well. spacey was designed by explosion AI, and the entire Purpose of spacey is to work at scale AI
at scale, we mean working with large quantities of documents efficiently, effectively and accurately.
spacey scales well because it can process hundreds of 1000s of documents with relative ease in
a relatively short period of time, especially if you stick with more rules based pipes, which we're
going to talk about in part two of this video. So those are the two things you really need to know
about NLP, and spacey in general, we're going to talk about spacey in depth as we explore it both
through this video. And and the free textbook I provide to go along with this video, which is
located at spacey dot python humanities.com. And it should be linked in the description down below
this video and the textbook I meant to work in tandem. Some stuff that I cover in the video
might not necessarily be in the textbook because it doesn't lend itself well to
text representation. And the same goes for the opposite some stuff that I don't have the time
to cover verbatim In this video, I cover in a little bit more depth in the video.
And in the book, I think that you should try to use both of these, what I would recommend
is doing one pass through this whole video, watch it in its entirety and get an umbrella sense of
everything that space you can do. And everything that we're going to cover, I would then go back
and try to replicate each stage of this process on a separate window or on a separate screen and try
to kind of follow along and code and then I would go back through a third time and try to watch the
first part Why talk about what we're going to be doing and try to do it on your own without looking
at the textbook or the video. If you can do that by your third pass, you'll be in very good shape
to start using spacey to solve your own domain specific problems. NLP is a complex field and
applying NLP is really complex. But fortunately, frameworks like spacey make this project and this
process a lot easier. I encourage you to spend a few hours in this video get to know spacey and I
think you're going to find that you can do things that you didn't think possible and relatively
short order. So sit back, relax and enjoy this video series on spacey. In order to use spacey,
you're first going to have to install spacey. Now there's a few different ways to do this. Depending
on your environment and your operating system, I recommend going to spacey.io backslash usage
and kind of enter in the correct framework that you're working with. So if you're using Mac OS
versus windows versus Linux, you can go through and in this very handy kind of user interface, you
can go through and select the different features that matter most to you. I'm working with Windows,
I'm going to be using PIP in this case, and I'm going to be doing everything on the CPU.
And I'm going to be working with English. So I've established all of those different parameters.
And it goes through and it tells me exactly how to go through and install it using PIP
in the terminal. So I encourage you to go through and pause the video right now
go ahead and install Windows however you want to. I'm going to be walking through how to install it
within the Jupyter Notebook that we're going to be moving to in just a second. I want you to not
work with the GPU at all. Working with spacey on the GPU requires a lot more understanding about
what the GPU is used for specifically, in training machine learning models. It requires you to have
CUDA installed correctly. It requires a couple other things that I don't really
have the time to get into in this video, but we'll be addressing in a more advanced spacey tutorial
video. So for right now, I recommend selecting your o s selecting either can use PIP or conda
and then selecting CPU. And since you're going to be working through this video with English texts,
I encourage you to select English right now and go ahead and just install or download
the N core web SM model. This is the small model. I'll talk about that in just a second.
So the first thing we're going to do in our Jupyter Notebook is we're going to be using the
the exclamation mark to delineate in the cell that this is a terminal command, we're going to say pip
install spacey, your output when you execute this cell is going to look a little different
than mine. I already have spacey installed in this environment. And so mind kind of goes through and
looks like this yours will actually go through and instead of saying requirement already satisfied
it'll be actually passing out the the different things that it's actually installing to install
spacey and all of its dependencies. The next thing that you're going to do is you're going to again,
you follow the instructions, and you're going to be doing Python dash m space spacey,
space download, and then the model that you want to download. So let's go ahead and do that right
now. So let's go ahead and say Python m spacing. Download to this is a spacey terminal command.
And we're going to download the N core web SM and again, I already have this model downloaded
So on my end, spacey is going to look a little differently than as it's going to
look on your end as it prints off on the Jupyter Notebook. And if we give it a just a second,
everything will go through, and it says that it's collected it, it's downloading it. And we
are all very happy now. And so now that we've got spacey installed correctly, and that we've got the
small model downloaded correctly, we can go ahead and start actually using spacey and make sure
everything's correct. The first thing we're going to do is we're going to import the spacey library
as you would with any other Python library. If you're not familiar with this, a library is simply
a set of classes and functions that you can import into a Python script so that you don't have to
write a whole bunch of extra code. Libraries are massive collections of classes and functions
that you can call. So when we import spacey, we're importing the whole library of spacey
and now that we've seen something like this, we know that spacey has imported correctly,
as long as you're not getting an error message, everything was in was imported fine. The next
thing that we need to do is we want to make sure that our English core web SM are small English
model was downloaded correctly. So the next thing that we need to do is we need to create an NLP
object. I'm going to be talking a lot more about this as we move forward. Right now, this is just
troubleshooting to make sure that we've installed spacey correctly and we've downloaded our model
correctly. So we're going to use the spacey dot load command. This is going to take one argument,
it's going to be a string that is going to correspond to the model that you've installed.
And this case n cor web s n. And if you execute this cell and you have no errors,
you have successfully installed spacey correctly and you've downloaded the English core web SM
model correctly. So go ahead take time, and get all this stuff set up. Pause the video if you need
to, and then pop back and we're going to start actually working through the basics of spacey.
I'm now going to move into kind of an overview of kind of what's within spacey,
why it's useful and kind of some of the basic features of it that you need to be familiar with.
And I'm going to be working from the Jupyter Notebook that I talked about and the introduction
to this video. If we scroll down to the bottom of chapter one, the basics of spacey, then you get
past the install section, you get to this section on containers. So what are containers? Well,
containers within spacey are objects that contain a large quantity of data about a text.
There are several different containers that you can work with. In spacey, there's the doc,
the doc Ben example, language, lexeme span, span group and token,
we're going to be dealing with the lexeme a little bit in this video series. And we're
going to be dealing with the language container a little bit in this video series. But really,
the three big things that we're going to be talking about again and again is the dock
the span and the token. And I think when you first come to spacey, there's a little bit of
a learning curve about what these things are, what they do, how they are structured hierarchically.
And for that reason I've created this, in my opinion, kind of easy to understand
image of what different containers are. So if you think about what spacey is as a pyramid,
so a hierarchical system, we've got all these different containers structured around,
really the dock object, your Docker container, or your dock object contains a whole bunch of
metadata about the text that you pass to the spacey pipeline, which we're going to see
in practice, and just a few minutes. The doc object contains a bunch of different things.
It contains attributes. And these attributes can be things like, like sentences. So if you
iterate over doc dot cents, you can actually access all the different sentences found within
that doc object. If you iterate over each individual item, or index and your doc object,
you can get individual tokens. tokens are going to be things like words or punctuation marks
something within your sentence or text that has a self contained important value,
either syntactically or semantically. So this is going to be things like words a comma period,
a semi colon, a quotation mark, things like this, these are all going to be your tokens. And we're
going to see how tokens are a little different than just splitting words up with traditional
string methods and Python. The next thing that you should be kind of familiar with are spans.
So spans are important because they kind of exist within and without of the doc object. So unlike
the token, which is an index of the doc object, a span can be a token itself, but it can also
be a sequence of multiple tokens, we're gonna see that at play. So imagine if you had a span in its
category, maybe group one are our places. So a single token might be like a city like Berlin,
but span group two, this could be something like full proper names. So of people, for example. So
this could be like, as we're going to see Martin Luther King, this would be a sequence of tokens, a
sequence of three different items in the sentence that make up one span, or one self contained item.
So Martin Luther King, would be a person who's a collection of a sequence of individual tokens.
If that doesn't make sense, right now, this image will be reinforced as we go through
and learn more about spacey in practice. For right now, I want you to be just understanding
that the doc object is the thing around which all of spacey sits, this is going to be the
object that you create. This is going to be the object that contains all the metadata that you
need to access. And this is going to be the object that you tried to essentially improve
with different custom components, factories, and pipelines. As you go through and do more advanced
things with spacey, we're going to now see in just a few seconds how that dock object is kind
of similar to the text itself. But how it's very, very different and much more powerful.
We're now going to be moving on to chapter two of this textbook, which is going to deal with
kind of getting used to the in depth features of spacing. If you want to pause the video or
keep this notebook or this book open up to kind of separate from this video and follow
along. As we go through and explore it in live coding, we're going to be talking about a few
different things as we explore chapter two, this will be a lot longer than chapter one,
we're going to be not only importing spacey, but actually going through and loading up a model,
creating a dog object around that model. So that we're going to work with container and practice.
And then we're going to see how that container stores a lot of different features or metadata
attributes about the text. And while they look the same on the surface,
they're actually quite different. So let's go ahead and work within our same Jupyter
Notebook where we've imported spacey and we have already created the NLP object. The first thing
that I want to do is I want to open up a text to start working with within this repo, we've got
a data folder. Within this data sub folder, I've got a couple different Wikipedia openings,
I've got one on MLK that we're going to be using a little later in this video. And then I have one
on the United States, this is wiki underscore us. That's going to be what we work with right now.
So let's use our width operator and open up the data backslash wiki underscore us dot txt.
We're gonna just read that in as F. And then we're going to create this text object which is going to
be equal to F dot read. And now that we've got our text object created, let's go ahead and see
what this looks like. So let's print text. And we see that it's a standard Wikipedia article
kind of follows that same introductory format and it's about four or five paragraphs long with a
lot of the features left in such as the brackets that delineate some kind of a footnote. We're not
going to worry too much about cleaning this up right now, because we're interested in not with
cleaning our data so much as just starting to work with the doc object in spacey.
So the first thing that you want to do is you're going to want to create a doc object