>> This afternoon's first session is--the title of it is visual analysis, in my--conventionally
in the past have been called document analysis, but I think this is a nice distinction.
It's that--it's looking a documents has visual objects, various papers doing this and trying
to extract information from the visual side of what is in the document.
Our first--our first talk today is from Brazil, it's going to be presented by Eduardo Cardoso,
and are looking at getting information from news webpage which is obviously important
class of WebPages.
And this is a--this is a group that--some people in this group have a long history with
Doc Joserio Rodriguez, worked with Luis Fernando Gomez Suarez, and--at Pukehio and the--and
he's been a contributor at various times in the document engineering community.
>> CARDOSO: Thank you.
Can everyone hear me?
Okay?
Good.
So, I'm going to present to you today "An Efficient Language-Independent Method to Extract
Content from News Web Pages."
It's a rather long title but it's very simple.
So, I'm going to start with the quick task overview.
Given a news webpage, we want to start by detecting the relevant content, which is the
middle portion where the actual real story is, and we want to go one step further and
extract structure data out of it.
So we're particularly interested in title, publication dates, and the news body.
Why is this important?
Our scenario is search engines, so we want to index documents for textual search and
it can also be use as basis to apply natural language processing and other task on top
of the--of the text.
There are other uses, but they're not our primary focus such as screen readers for the
visually impaired and reflowing for small screens.
A quick recap of the state of the art, page level approaches are very common, there are
very good results for most labels, by labels, I mean title dates, body, some works talk
about author, advertisement, etcetera.
But most of them rely on page rendering, which for our scenario is too slow.
So, our desirable characteristics are broad applicability, which means a page level approach
and we add one restriction that we want to be able to apply this to a document written
in any language, English, Portuguese, Spanish, even Chinese.
We want a high extraction quality, that's very important, and performance.
We're dealing with a high volume of documents, so page rendering is too slow and we need
something faster.
We're going to talk a little about our approach.
We use a machine learning model to classify DOM text nodes, and we use visual features,
but since we don't use page rendering, we wrote our own CSS parser, and I'm going to
talk a little about that ahead, and again our focus is on large scale document processing.
So, I will start identifying the relevant content.
And then, we proceed with title detection, date detection, and body detection in this
order.
To detect relevant content we use the NCE algorithm, which is a previous work of ours.
It was published in 2009.
I'm not going to into details of how this algorithms works, but basically we look for
a massive portion of text and the page and then we proceed by refining it.
And other algorithms could be use here such as readability, which is well-known.
And once we detect the relevant content, we will be working on a subset of the DOM nodes
in the document, and the subset is based on the relevant content.
We expand it a little bit, but focus is on the relevant content.
So, from now on, everything happens on this subset of nodes.
We have a small set of features, and I'm going to highlight the ones we feel almost important
for each step.
For title detection, the edit distance from the text node to the document's title tag
is often very, very useful because they often overlap.
The font size is very important the title is usually displayed in bigger font sizes
and we use the number of similarly styled nodes in the document.
This deserves a little explanation, when you look at a news page, we can often identify
the title very quickly, even if it's in the language we don't know how to read or speak.
That is because it's usually presented uniquely, it stands out from the rest.
So, we look for a combination of font size, font color and bold text, and we count how
many nodes present this combination in the document.
And titles are usually unique in this regard so this feature usually has a value of one
or two for titles.
And if we can find a suitable, title we just go with the document's title, the title tech.
For detection, we usually look for small nodes with a small amount of text, and we look for
the presence of digits.
This is because usually we have a time stamp such as 2:00 PM or a date such as September
21st, and we see numbers there, most of the time.
So, it's usually a good signal.
And we look at a distance to the identified title from the previous step.
Usually when you look at the title and a date in the dormitory, they're usually close, so
this is a good signal to detect dates and we look for nodes close to the title.
This has a small issue that our model becomes title dependent, so if we happened to miss
the title or date detection won't be as good.
We perform some experimentation and this didn't show to be a very--the results were still
good.
It wasn't as bad as we thought and we just kept it because it was looking good.
And for body detection we're going to use the relevant content that the algorithm provided
us, not the expanded subset, and we're going to remove the title and the date if they are
contained in the relevant content.
That's going to be our body.
We tried some other approaches but they didn't really show much improvement.
So, I'm going to talk a little bit about the quality and applicability of our method.
We use two corpora, the first one, we created ourselves, it has 200 hundred pages written
in English, Portuguese, and Spanish.
And then we use another corpus which has works published in electric chair.
I believed it was presented last year, Document Engineering as well, is the news600.
And it has about 600 pages all written in English.
So we started with the--an intra-site cross validation, what we called intra-site is actually
we trained the model on some pages for instance pages from CNN.
And when we apply it, we applied to other pages also from CNN.
So, the model is able to capture some characteristics from that website and the results we have
are about 92% of that one for title, 84 for dates and 87 for body.
And we also did what we called an extra-site cross validation, which is the opposite, we
train using CNN but we only apply to BBC, Fox News and other websites.
So, this shows how well our approach can be used for pages that we didn't intend originally
for--to be used on scene scenarios.
So we still have 91 percent for title, 77 for date and 87 for body.
I have the slide here with the difference from one to the other and we can see that
title and body have very stable results.
We can capture these two labels with good results.
And we see it some fluctuation on date, and we attribute this to the fact that we don't
use textual features to detect dates.
So we don't have a dictionary of days of the week or months, because we are interested
in applying this to any language, so we didn't use that.
We also try to apply it to some pages in Chinese but we did not have a notation for it, so
we don't have any numbers.
But the visual--the visual inspection that we conducted showed some consistent results.
And then we applied to news600, which we didn't use until the approach was finalized.
So this is--this is good to check if we're biased by or choice of pages, and it seems
we're not because the results were so good.
We have 93 percent for title, 86 for dates and 88 for body.
So, I talked about quality, I talked about how our method can be applied to several pages,
I'm going to talk a little bit about the performance which is the part we're most interested.
We use the subset of DOM nodes, a subset of nodes from a document, and we were very conservative
when using the subset and we didn't discard as many nodes as we could.
And we still manage to discard about half of the nodes in the page.
And when we look at the time impact that this has, it cuts our evaluation of nodes about
by half which is--looks good, it's proportional to the amount of nodes we discarded.
So if we were more aggressive, we could get better running times.
Since we were with our own CSS parser, we were able to optimize it as well.
Not all rules affected our features.
So we were able to focus only of those that actually had any impact.
And those were about 30 percent of the rules we found.
And when we looked at the time impact, we could also reduce it proportionally.
We could then combine these both--these two optimizations and our total execution time
dropped to about half that we had, it's the left graph.
It only dropped about half because we have some test that couldn't be improved here such
as I/O and the HTML parsing which happens in both methods and that data structure initialization,
etcetera.
And on the right we can see our method compared to an equivalent method that instead of our
HTML parser and our CSS parser uses a rendering engine such as WebKit.
And this is a very optimistic approach because we're using WebKit, without images, without
java script and without plug-ins.
Sometimes, these are important but in a best case scenario, you have all of these turned
off and we can see, we can run in about 16 percent of that time, which is very good when
you have a large number of documents.
So, to conclude, we have stable results in a variety of pages.
We have a good extraction quality.
Some methods have better numbers than ours but they don't have the same performance that
we do and it's a trade off and we feel we have good results given the constraints.
And we--we're able to use visual features with much faster running time than when using
a full rendering approach such as using WebKit.
And we plan on extending this work in the future to other task on web.
And that's about it.
Thank you very much and if you have any questions...
>> Are there questions for Eduardo?
>> Because I'm curious, you talked about performance and so both in my PhD research and in then
in my first job afterwards, we did content analysis in PDF documents, it's totally a
different class.
Seems today...
>> Face it--face it [INDISTINCT] redirect the...
>> I will--I will speak louder.
Okay.
So, yes, so, in my past research and in my actually work, I've done sort of content analysis
and content recognition.
We have much, much higher percentages of accuracy on more complicated document forms including
newspaper and magazines.
We trained models that analyze sort of indexing and positioning and things.
So your reasoning behind this was performance, you said that you can basically discard all
of these information and just look at certain characteristics and try to determine that
they--to use that information to go public about the document.
One thing I would ask is, when you don't have something like HTML and CSS, it's reasonably
trivial to calculate font properties, if you look and see a post-script model or something
in a different font--document category, where you have to basically render the document
to get a lot of the analysis out of it.
Where did you get benefit from this algorithm and how does this scale to the problems, this
is the one I'm trying to get at...
>> You mean... >> ...given the--given the reduction in accuracy.
>> How we could apply this method to other documents such as PDF?
>> I'm trying about--I'm trying to understand how the scale, in terms of performance, as
it moves performance that answers amenable to the sort of analysis you're doing.
And that you lose quite a bit of accuracy on [INDISTINCT] algorithms I've used in the
past on analyzing documents and I'm trying to understand why you--why you made that trade
off?
Why you feel the performance is such a key characteristic?
>> Okay, it's actually--I think a two part answer.
The first one we had some technical limitations on project, so we were using a decision trees
as our machine learn method.
And we did some quick experimentation later.
And changing the method to something like an SVM and maybe tweaking some parameters,
we were able to achieve better results.
This would be--there's room for improvement here.
Just changing the method but I'm not really sure--you mean scale need to other documents--I
didn't really get this part.
>> In analyzing perhaps?
>> Yes, that's--for the--but they're determined.
This HTML and CSS is very amenable to this type of analysis.
A post-script based document for example probably wouldn't it be old, it will be harder to produce.
And so the traditional sort of glyph analysis and measurements and the styling characteristics
of those--and sort of broad techniques of bringing sort of multiple sources of data
to produce sort of very high levels of content recognition.
But they all are a lot slower, I'm trying to understand why the performance was such
a--why you felt there was need for that?
I'm just didn't really understanding what you are trying to achieve over all.
>> Yeah, we need the performance because we are dealing with a lot of documents and we
need to process them quick.
And I think the performance is very particular to the scenario of HTML and CSS.
Since we are able to discard most of the document, we are able to look at small portions of it,
and only the bits that interested us.
So, that's where the performance comes from.
The approach probably works if we calculate same features for instance, on a post-script
PDF file.
It probably works, we haven't experimented with it but we--the performance wouldn't be
the same.
We would have to do some research on that specific that type of document.
Does that answer your question?
>> I think you did.
>> Yes.
We can talk later.
>> Other questions?
Okay, let's thank Eduardo for his presentation.
>> Thank you.