[Music]
If you have ever studied computer
science or programming
you'll probably experience the pain of
writing code on the paper
for an exam quiz or whatever you have to
deal with the unbearable uncertainty of
whether it will work or not.
From that reason I decided to create a
Python project named
"Run On Paper". Also unlike my other
projects
I decided to develop it on the video so
you guys can see the whole process.
Anyway main goal of this project is to
detect routine code from paper and
execute it. There are two main challenges
with this project,
first recognition and second execution.
I will go over them both while coding, so
let's get started.
First we need to make sure that few
things are installed.
To recognize characters from the image
we need to install tesseract,
which is an optical character
recognition engine. It will hold our main
recognition logic.
Also in this project we will be mainly
focusing on
reading and executing java code, because
java is generally the preferred starting
language for beginners.
So make sure that it is set up on your
pc as well.
Now as you can see i have pycharm open
right there
so let's create a new project and name
it run on paper
once that's done we can go ahead and
install necessary modules
which are opencv and pytesseract we use
opencv to capture images
and make basic image processing we use
pytesseract as a middleware to connect
with the tesseract so now they both are
installed we can start coding
let's create a new python file named
main.py
inside that file we add the necessary
import statements
import cv2 import pytesseract and finally
import os
we import os because of the read
write operations that we needed to do
after we have done that
we have an optional line. If you defined
your tesseract.exe
under your environment variable paths
you don't need to add this
if not add it otherwise pytesseract
won't work
[Music]
we are done with setting up our modules
now we are ready to capture from our
webcam
to do that we need to add this line it
will initialize the video capture
then we enter an infinite loop and
capture from the camera frame by frame
notice that we have two parameters one
is ret
and other one is image. Image is pretty
self-explanatory
it is captured frame and the ret is that
frame was successfully captured or not
It is basically a boolean value
after we have captured our frame we
display it on the screen by using
opencv's
imshow method now things can get a bit
different
at this point we need to keep the
captured frame or get rid of it
it is essentially a taking snapshot from
live feed
so we implement this logic by taking an
input
if that input is space then we can use
it
if not a new frame should be displayed
once we have picked our frame we need to
improve it
so that tesseract can detect characters
much better
to do that first we need to convert our
image
to rgb then we add a threshold
which you can think of it as black and
white filter on
applications like cam scanner if you
have taken a picture
from a paper it is generally has a
yellowish tint
this process turns that into
pure white
same also applies for darker colors as
well
after that we denoise the image since my
webcam image quality is really poor.
We do that by adding this line now our
image is ready for recognition.
Recognizing text in the image is really
easy with tesseract
all we have to do is call the method
image to string
then we pass our image as an argument it
will return the recognized text
by doing this we are done with the
recognition part
now we can move on to the execution.
In this part what we need to do is grab the
recognized text
write it into a java file compile and
execute it.
Well that seems easy, so let's do that.
First we create and open up a java file
we do that by using the open method it
takes two arguments
file name and mode in our case let's go
with the w mode which is basically
overwrite the existing file if it has
any content in it
you can also make it to append instead
by passing the mode value as
a after we have created our file then we
can write it using the write method and
pass our recognized text
that is pretty straightforward then
finally we close the file stream
we we have now finished the capture
logic rest is actually pretty simple
we add quitting button release the video
and destroy all windows
at that point we can call our batch file
which we will be creating in a second
to call and execute our batch file what
we need to do
is to use the os module and use the
system function inside that
it is as simple as that now we can
create
our batch file
[Music]
inside our batch file we first add our
status output by using echo
then we add compilation methods for java
which is
javac followed by our java file name
main.java
after we do that we can easily execute
the compiled binary by using the command
java followed by class name which is
main in our case
now it should execute our recognized
java program
also on a side note if you are on linux
or mac you can use shell script instead
of batch script
it will make things much more easier
anyway
now we can test our program let's run it
and see if it works properly
[Music]
as you can see i do my best to write as
bold and as clear as possible
in order to make our program recognize
however it is not enough
so i go ahead and print out the text but
it doesn't work properly as well at this
point i probably need to improve the
trade model by possibly training my own
but as for this video to prove my point
of execution let's simplify our code
just get rid of all the video capture
stuff and make it accept only a single
image that we provide to it
and as for the image we open up our
notepad write the sample java code and
take screenshot of it
now let's try one more time as you can
see this time it worked as it should be
like i've mentioned previously if you
train a custom model for this specific
application
the results would be better also with
the idea on our hand we can do much more
cool stuff like turning it into a mobile
app
and possibly adding an AR to display
the execution on the paper as well
but improving, that is on you :) Anyway,
I hope you enjoyed the new style
if you like it please make sure to like
button and consider subscribing
see you next time take care!