When you're building NLP application, you
might end up using multiple libraries
such as spaCy, NLTK
Gensim, PyTorch, TensorFlow, Hugging Face
and so on. SpaCy and NLTK are
two more most popular libraries
that are being used in any NLP
application, and they tend to do similar
things.
So Baby Yoda is asking if they're doing
similar things then what's the
difference? Well in this video, we are
going to cover the difference between
these two popular libraries.
See he's happy!
The very first difference is that spaCy
is
object-oriented, whereas NLTK is string
processing library. Now you'll be like
what does that mean? Well let's try it
out! So I'm going to install both the
libraries and show you a sample code.
I have installed Git bash here. So if you
have not installed that on your machine,
you can just run
like you know download Git bash. So Git
bash allows you to run unix commands on
Windows. You can do it on Windows command
prompt as well, it's your choice. And I in
I open
Git bash from here. So it's just a
command prompt right? You can do the same
thing from Windows command prompt also.
You'd say pip install
NLTK, and I'm assuming you have installed
Python already. So you need to have
latest version of Python installed. So now
that install NLTK,
you can now install pip
install spaCy
and that will install
spaCy library as well.
Okay it will take some time, but it will
be
installed pretty smoothly. You you need
to also download some packages for spaCy by
running this command called
python
-m spacy
download so you are downloading all the
packages
related to en en is English
So for English
language specific spaCy models,
see it's downloading some spaCy models
that you'll be using um,
to write your code.
After you have done that,
you can run
jupyter notebook
or the other way. So
I'm assuming you you know what jupyter
notebook is,
and you have
installed jupyter notebook on your
computer,
and you can run jupyter
notebook by running this command.
And it will launch
an interactive
cell, you know in in a browser where you
can write your Python code.
I am running this in this particular
library, uh sorry in this particular
directory. So make sure you enter into a
specific directory because in jupyter
notebook you'll be able to see the code
of
just that directory.
And
I'm going to, I have created a folder. You
know doesn't matter you can do this in
your root directory and you will say new
python 3
okay?
So
that is creating
the
new notebook
and
now what I'm going to do is,
in spacy I will do sentence and word
tokenization.
So here,
import spaCy library you can do it by
doing this ctrl enter to execute it, and
b b shortcut to create a new cell.
And you will
say,
spacey.load
So this
loads a specific package. So I want to
download,
I want to start using
English,
like the English core package for this
particular use case. And you will call
this object NLP. By the way we will go
into more details later on. But right now
I want to explain you the difference
between spaCy and NLTK,
and I want to show you by writing this
code,
why spacy is object oriented and why
NLTK is string processing why it is not
object-oriented okay?
So once NLP object is created, you can
create a doc and supply a string. So
okay, now I'm going to supply some string
which you are going to like it.
So see Doctor Strange loves pav bhaji of
Mumbai and Hulk loves
chat. Delhi chat.
So these guys visited apparently India
and they fell in love with the street
food,
and here is the document because it has
two sentences correct? One
and two.
And we saw in a previous video that
it's a usual practice that
you have a paragraph and you want to
split the paragraph into individual
sentences, and then individual sentences
into individual words for your NLP
application.
It could be any use case, we'll look into
this use case later okay? Don't worry! But
just think,
you are building NLP application. You
want to split a big paragraph into
sentences and this is how you do it in
spaCy. You will just say for
sentence
in
doc.
by the way you can hit tab also, it will
auto complete.
print
sentence
See, it printed two sentences. So it split
these 2 sentences
accurately.
And it is a little smarter, it is not
just using dot because if it was using
just dot
you will get Dr as a separate statement.
But here it understands that Dr. is
part of one big sentence and it is not a
sentence in itself correct? So see that's
that's where you see the
benefit of using these packages.
Otherwise, I can write Python code and
split everything by dot right? But if you
have things like
etc. you know you your Python program
will suck because it will start
splitting sentences by every dot. So you
need something smarter.
So this is basically sentence
tokenization in spaCy
and you are already realizing that
we are
using object-oriented programming here
because we created this object and now
object has a property called sents and
we're just iterating over it. So see it's
easier to write code this way.
If you want to
separate out individual words from these
sentences, what you will do is, see it's very
intuitive. for word in sentence It's
like English.
It's like I am writing English
and it just
prints individual words from all these
sentences.
So this was a sentence tokenization, this
was a word tokenization in spaCy and by
looking at the code, you can say this is
using object-oriented programming.
Now let's do the same thing
in
NLTK.
I would say import NLTK
and in NLTK,
if you want to
tokenize,
so you will say
from nltk.
tokenize
import hit tab
See there are so many tokenizers
tokenizer,
and that's the thing
that
NLTK offers
a lot of customization. You can choose
which algorithm you want to use
okay? So maybe I can
go to
the differences that we have. I have
created
a
presentation here. So let me
present it to you.
So the first difference we saw was
spaCy is object-oriented and NLTK is
mainly a string processing library okay?
Uh
spaCy provides most efficient algorithm
for a given task. Whereas in NLTK you
have an option, it's like using a manual
camera versus
uh prefix setting. So spaCy is like your
mobile phone. You know you have fixed
setting, you take a picture the picture
is usually very good. NLTK is more like a
manual DSLR camera which gives you a lot
of control.
But then you have to select specific
settings
okay?
So that's what you're seeing here, that
here
when you hit import, you can
import so many different tokenizer okay?
I am going to,
I'm going to import
sentence, like the default
sentence tokenizer,
and I will say
send tokenize
the string
The string is nothing but
this.
And by the way if you
get any error such as
punct is not downloaded, etc.,
you can,
you might if you're using it for the
first time, you might get an error. And
you will have to do
something like
NLTK.download
the default package you know
punkt So you do run this code
if you are getting error at this point.
You can also
just run this code and let's see what
happens.
So when you,
when you run this okay,
let me show you
you will get this kind of window.
And this window
tells you that
in NLTK when you did pip install NLTK
it installed the base library. But it did
not install
all the packages related to different
languages, related to different
tokenization schemes, and so on. You can
install all of them manually by by
running NLTK.download it will open
you the screen and you can say okay,
all packages download or you want to
download specific packages.
Your models you know like specific
models, like word to wag
these are like specific language
specific models that you can download.
You can also download a different
corpora
okay? Like for different corpus
like WordNet,
Treebank, Twitter Samples see.
You can download all of these
as and when need
arises. But for now most likely you will
need to
run this sentence, this sentence just
once and it will work. Now see
here
this sentence tokenizer is not working
really well because it said doctor is a
separate sentence. See it's an array with
3 sentences: 1,2
3
So it did not work well in doctor and
we'll we'll look into this later. But
the purpose of today's tutorial is to
show you that
NLTK is a string processing library.
See there is no object here. There is no
object-oriented programming. You give
string as an input, you use you get
strings as an output.
Same things with
word tokenize.
If you do word tokenize
of
this big statement
it will split the words
from that statement,
and again it's not very perfect doctor
dot I'm not saying NLTK sucks here
because see this this got it right in
the first sentence. What I mean to say is,
NLTK is like a DSLR manual camera. You
need to tweak.
It can be really powerful library. People
use it in production, in big companies.
But you need to
tweak it
to
satisfy your needs. Whereas spaCy out of
the box,
it gives you the best algorithm, best
option and it works
okay? So just by looking at this code you
realize spaCy is object-oriented. NLTK
again, see word tokenize you give string
as an input, you got string as an output.
And that is a reason why here I'm saying
is NLTK is mainly a string
processing library. The other difference
is spaCy is user friendly. NLTK is
also user friendly by the way. But just
my personal opinion, I find spaCy to be
a little more user friendly than NLTK. So
that's the second difference.
spaCy
doesn't allow you to select specific
algorithm. See when you're doing
tokenization, you can do tokenization in
5 different ways.
SpaCy will select the best way for you.
It's like taking picture from cell phone
you know.
It just takes a base picture for you.
Whereas NLTK, you can customize, you can
choose which algorithm you want to
select. And hence, like spaCy is a
relatively new library, and it is
on many occassion you will find it is,
it performs better than NLTK.
Now since NLTK allows you to do a lot of
customization, it is it is perfect for
researchers.
Whereas spaCy, if you want to get things
done,
you know you don't worry about too much,
about the algorithms and all that.
Go for spaCy because it is based for
app developers!
spaCy is a new library, has a very
active
community. Whereas NLTK is
old library, and the community is not as
good as NLTK. And here is the summary of
the differences. I hope you like this
video. If you did please give it a thumbs
up! I'm putting a lot of afford in making
this tutorial. So you can share it with
your friends, especially share this
entire series on LinkedIn if you're
liking it so far.
In the next video. we are going to come
up with a new topic. Hopefully, we'll
write more code. Thank you!
[Music]