WALE AKINFADERIN: My name is Wale.
I'm a GDE in machine learning.
During the day, for my day job, I actually
work in the intersection of machine learning
and renewable energy.
And then during the night, I research
machine learning for the developing world
and also AI for social good.
And today, I'm going to be talking
about how I'm using NLP and machine learning
to understand parliamentary bills in low resource
countries.
I think the first thing I'm going to talk about
is what a low resource country is.
So low resource countries are countries with low--
a low-income economy and also low and middle economy.
And this name is coined by the World Bank and the United
Nations.
So one of the motivations for my talk
came from the Open Data Barometer
for Sub-Saharan Africa.
So this is the data that is annually published
by the WWF Foundation.
And one of the things I saw is that, in Sub-Saharan Africa,
the average score is actually 14 out of 100.
And this is based on the readiness, some implementation,
and the impact.
And then I actually went to Nigeria,
which is where I originally come from.
And I saw that the score is actually 21 out of 100.
And they are ranked 78 in the world
out of about 110 countries.
And then going further, I actually
also check the United Nations SDG.
And one of the SDG goals that caught my attention
is SDG 16, which actually talks about peace, justice,
and strong institutions.
And then I also looked at the SDG target and some
of the targets, which one of them
is to be able to develop some very effective and transparent
institution, and also to make sure
that, when there a decision making in government,
that everybody is included and there's also
some sense of responsiveness in that.
And this led to a problem statement,
which means, how can we make sure
that when parliamentary bills are introduced in low resource
countries, that the process--
that the process of creating introduction and also
the process where they want to analyze and process data.
The average citizen can actually go
and say, oh, I want to check the parliamentary bill,
for example, from my constituency.
And I also want to know what category this is.
And the objective of this project
is for us to be able to generate some semantic and unstructured
data from the views.
And also we can categorize them into some labels that
are socioeconomically driven.
And the first thing we did is that we built NASS-AI.
So NASS is the-- the NASS stands for the Nigerian National
Assembly.
And what we did is we actually went from the Fourth Republic,
which is from 1999 to 2018.
And we scraped all the data from some website
and some back server.
And then we got all the bills that they have introduced.
And we actually got about 2,400 bills.
And what we saw was that the bills,
they have different variety of quality.
So you have this--
this first one, which is a pretty good bill.
The text are embedded in them.
So the quality is good.
And then we have another one, which is not so good.
This is probably from maybe 2006.
And we have this last one, which is really terrible.
And these are some of the data sets that we have to work with.
And I'm sure you guys understand that trying to clean and build
a data set is as simple as that.
Maybe not.
And so this first bill is a bill to regulate local government
elections in Nigeria.
And the second one is a bill to prohibit the use of military
during a live protest.
And then the last one is a bill to provide
free screening and treatment of cancer
to the citizens of Nigeria.
So what did we do next?
We took this bill.
And we labelled them.
And we made sure that we chose eight different labels that
actually represent all the facets of government.
And also they're really socioeconomically driven.
So for example, we have energy, environment,
and natural resources.
So one of the things we saw, we had a bill
that actually wants to clean up the oil
spills in the South-South region of Nigeria, for example.
And then we thought, is this an environmental bill or energy
and petroleum bill?
And we noticed it's the two of them.
And then we actually put them together.
And so this is the distribution of the different labels
that we have.
And then we can see that it's actually pretty in balance.
And the first thing we did is to take this bill
and put it into an OCR too.
Because of the nature of our data set,
we have to force some of our OCR choice.
So the first thing is clean bills with embedded text.
We could put them into PDFMiner and extract some of text
from it.
The ones that are not so good, we converted them
to images using PDF2Image.
And then we used the pretrained OCR,
the PyTesseract to extract the text from it.
And then the next thing is the preprocessing.
So we took these bills.
We removed numbers from them.
We changed them to lowercase.
And then we tokenized them.
And then we removed the stop words.
And eventually, we tried to lemmatize
to get some of the important words.
And then we took this output from the OCR
into a document embedding.
So that means we used Doc2Vec to put them
into some sort of vector.
So we actually use the PV-DBOW, so the Paragraph Vector
and Distributed Bag of Words.
And for us to have the different class,
you could see the embedding space on the right-hand side.
And so this is what our methodology looks like.
The first thing is we take the bills.
OCR tool, we preprocess like I mentioned,
goes into a document embedding.
And the best results we got so far
is actually from a bidirectional LSTM.
And so we have--
it goes through the forward LSTM then the backward LSTM.
And then the output from this LSTM memories,
we concatenate them then to dense layer.
And then this Softmax layer is to classify them
with eight neurons, which stands for eight classes.
And we use LSTM because you could actually-- going forward
and backward, it could remember some of this memories better.
And after implementing this, this
is the result for a Doc2Vec and bidirectional LSTM.
We had an F1 score of 0.071.
We had very good results.
We held on agriculture.
And I think we label sports and social welfare
is not very good.
And we have a coefficient matrix there.
So we also tried a lot of methods.
And I'm actually showing about three of them
to just show that Doc2Vec and BiLSTM is better.
And we also tried Word2Vec and a multilayer perceptron.
We tried TDIDF with SVM because we
know SVM is very good for high dimensional data set.
And so for Doc2Vec, and BiLSTM actually
beats all the baseline methods.
And so in conclusion, our experimental results already
showed that our model is effective enough
to categorize this bill, which will eventually
aid our large-scale digitization effort.
And also this-- my idea is that this result should
have substantial impact for the different low resource
countries and also for them to be
able to meet open data charter principle,
not for them to have some bills at the back of a server.
For them to be able to get these bills,
have it publicly, and be able to categorize them properly.
So what are we doing next after this?
Well, we are actually writing--
we have a paper that we're going to submit to NeurIPS Machine
Learning for Developing World workshop that
comes I think in December.
And we're planning to extend this
to three other countries in Sub-Saharan Africa.
So we have Ghana.
And we have Kenya and South Africa.
And the choice of these countries
is actually very strategic.
One is a lot of-- they are actually
English-speaking countries.
So it will make it easier for us to generalize our models.
The second one is they have better metadata.
So one of the things we want to do eventually
is to see if we can build a very good knowledge
representation of parliaments in Sub-Saharan Africa.
So that means in Kenya, for example, they are going to--
we can get data apart from the bills.
We can also understand bills that actually
went into first reading, second reading, third reading
before we can--
before it actually passes.
And hopefully, we can build a model to sort of say, oh,
this is--
we have some probability that this bill
is going to go into the next reading or the next phase.
And also we also want to build a custom OCR.
We're thinking we're going to get people to label the data
and also build something that we can extract very good data
from the low-quality bills.
And I think last Friday, we got--
we got some grants, which is from the AI for Development
African Innovation Grants that will help
us get people to work on this.
Thank you.
[APPLAUSE]
[MUSIC PLAYING]