So, now that I'm in quarantine,
I have basically this one hobby, which is going for walks.
I do everything on walks, I listen to podcasts, I take meetings.
There's just one thing that's really hard to do on a walk
and that's reading a research paper.
Or so I thought until I learned about a new project that my colleague, Kaz Sato,
who's based in Japan, is working on.
He used machine learning to transform PDFs into audiobooks.
Today, I'll show you how to do the same thing,
how to use computer vision and text-to-speech
to turn your own PDFs into audiobooks.
Let's get started.
♪ [music] ♪
So first I decided to chat with Kaz to see how he built his project.
I have converted the PDF books into the text file
by using the Vision API OCR feature.
I have used here Auto ML Tables to understand the layout of the document.
And then I have used the Text-to-Speech API for converting
the pre-processed text into the .mp3 audio file.
I wanted to build my own version of Kaz's project
that would convert research papers into audiobooks.
So I'm going to copy most of Kaz's architecture.
First, like Kaz, I'll create a Google Cloud Storage bucket
where all the PDFs I want to be converted will be stored.
Next, I'll use the Vision API to extract text from those PDFs,
which I'll later convert into spoken words.
I'll use Kaz's code which calls the Vision API from Python.
Here's what the output looks like when you pass a PDF through the API.
As you can see, it contains both the raw text
but also the X, Y coordinates of the characters.
So Kaz set up this project in a really nice way.
He used a Cloud Function, which is a way of running a tiny bit of code
in the cloud when something happens.
In our case, we'll set it up so that uploading a PDF to cloud storage
triggers the cloud function that kicks off our audiobook-making pipeline.
I'm going to create a cloud function, too, and copy Kaz's code into it.
After we've converted the PDFs into text, we still need to decide
which parts of the PDF text we want to include in the audiobook.
For example, we probably want the body text and the title,
but we probably don't care about things like page numbers
or references or image captions.
So this is where our custom machine-learning model comes in
to do what Kaz calls remove the garbage text.
The Auto ML Tables model can detect whether
a text could be a garbage text,
such as the page headers or the page numbers,
or the body text, or the other small labels you can find in the diagrams.
So to handle the problem of identifying the garbage text,
Kaz built a custom machine-learning model using a tool called Auto ML Tables.
To train that model, he collected all this information
about the size of the text on the page and its position.
But in order to do that, Kaz had to manually label a lot of PDFs,
marking the good text from the garbage text.
It took three hours for each book,
and for the initial five books,
it took like two days.
That's 15 hours. (laughs)
I could do that because I was back in my parent's home,
and I had nothing to do except for this one.
So in order to avoid labeling data myself,
I'm going to use a little cheat here.
Instead of building a custom model, I'll just look at all the font sizes
and whatever font size is most frequently used,
I'm just going to assume that's the body text
and that should go into the audiobook.
If you want more details about my and Kaz's approach,
make sure you check out the blog post linked below.
And now for the fun part: generating computer voices!
(computer voice 1) I'll take it from here.
Google Cloud Text-to-Speech supports over 220 voices in 40+ languages.
Let's try another one.
(computer voice 2) I am your narrator.
(computer voice 3) I am your narrator.
(computer voice 4) I am your narrator.
(computer voice 5) I am your narrator.
(computer voice 6) I am your narrator.
Perfect. Now that I've chosen my favorite voice,
I can update the codes,
that when I upload a PDF, an audiobook will automatically be generated.
Let's take a look at what it sounds like.
(computer voice 1) A promising path towards autoformalization
in general artificial intelligence.
(computer voice 2) Christian Szegedy.
Today, AI systems are able to learn solving tasks
that used to be thought of taking
uniquely human capabilities, until recently.
(Dale) Thanks for watching.
Let me know in the comments what you want to see me build.
See you next time.