what's going on everybody and welcome to
another intermediate Python tutorial in
this tutorial what we're going to be
talking about is async IO or a sync EO
depending on which way you want to say
it I won't judge other people will
and what async IO is for is for
asynchronous programming and probably
the best way to kind of understand
concurrency in basically any programming
language is it's often confused with
doing something in parallel it's kind of
like you're you're handling maybe
multiple things in parallel but you're
definitely not doing them you might
you're aware of them in parallel I
suppose but you're not doing them in
parallel so for example like let's say
you're you're all out for a jog and you
feel like a little tap on your shoe you
look down your shoes untied
do you have to finish your entire jog to
tie your shoe I hope not right like that
would be that would be unsafe - all
right if you're very unpleasant and
unsafe at the same time so no you you
stop your run you suspend your run and
you bend over you tie your shoe you
finish tying your shoe and you get back
to your run ok and this is you you're
doing these two tasks right right -
you're able to tie your shoe and you're
able to run concurrently so the same
thing is true with programming like
let's say you've got a web page and that
web page needs to load a whole bunch of
elements but you know as it's going down
the page and it gets to some image that
it's requesting from some server that
server just is not responding what
happens well if your page is loading
synchronously well your your entire web
page will hang but if your page loads
asynchronously then the other elements
are going to get to load and you may or
may not ever love that image maybe that
server is down or something but if it's
just taking a long time while you have
that latency between the the i/o and
hence async IO input/output while you
have that delay you could be doing other
things other than just sitting there
literally doing nothing and waiting on a
response
now what I'd like to do here is just I
think we're just gonna start with a
really basic function and then we're
gonna turn it into a Co routine if I say
go routine in this tutorial forgive me
uh my first experience with concurrency
was actually with go routines and go and
so now I just call it co-routines go
routines you might not even notice
anyways let's just say this is the
function that we're going to use so all
it is is it's just gonna find some
numbers and by the way yes I'm just copy
and paste of this function it I'll put a
link in the description to the
text-based version and you can get to
grab this function or you can write it
as I'm talking if you want but basically
the idea of this function is to just in
any range we're going to divide by
something so then what we're gonna do
when we start the function is we're
gonna print out what we're doing what's
the range what's the div by we're going
to start allocated so anything in that
range that's the basically that's
divisible by whatever our div by is
we're gonna add that and then when we're
done we're just gonna have a little
thing that says we're done and it
returns whatever we located now to run
this I'd probably just put it in a block
but for the purposes of this tutorial
we're going to go ahead and we'll use
define and we'll just have a main a main
function here and we'll just say divs 1
equals and let's just copy and paste
that and that will be just so I match
the text base right up let's do 508
thousands 34 1 1 1 3 3 and then I'm just
going to copy paste paste make that two
three one thousand fifty two thirty two
one zero and then we'll do
hundred three okay so now we've got our
main function here and then just for in
case someone wanted to import this
amazing script if name equals main we
would have just let's just run the main
function so so with that you also you
know you you can change this function to
be anything you want you can you can
have it doing Fibonacci stuff if you
really want to be like everybody or you
could do anything you want really I will
say if you are interested in doing truly
you know HTTP tasks you can look into AI
Oh HTTP for doing asynchronous
requesting and stuff like that
so that's why we're not going to use
that and for this example but just take
note so now that we have this just want
to do okay what we want to do is we
could even let's just run this real
quick just see what we're dealing with
so as you can see this whole script runs
its about 0.3 seconds and we can see it
runs in the order that we called it
basically so runs divs one it finds it
just starts iterating through five
hundred and eight thousand finding all
instances where we have a number that's
divisible by 30 4113 so it does this one
and then it does this one and then it
does this one so what could we do to
convert this to be asynchronous and why
might we want to do that well for
example what if this was instead this
right and we ran this it's gonna hang on
this one where we actually really could
have found the answers to these two and
then come back to this one and spent
about the same amount of time alright we
didn't have to waste all this time still
waiting on this one right and so so
that's kind of what we're trying to
solve we're trying to figure out a way
that we don't need to do this and we'd
ideally not like to use something like
threading or multi-processing if we
don't have to because that adds in a
whole slew of other things that we might
may or may not be interested in sealing
so so so at this point this program is
totally synchronous it runs in order and
all that and now what we want to do is
begin to bring this to bring async IO
into this I'm just going to bring over
the documentation again in the
text-based version of this tutorial I do
put a link in here but as you can see
this is a pretty large package so
there's a lot going on here in my
opinion I always wish in the standard
talks they would have like a you know
stupid simple section just first like
super basics and then you could go
through this stuff because that's
basically what we have to do every time
we find something we have to do like a
really simple example and then and then
we can and then we can maybe come to the
docs and see what what can we tweak and
change and all that but anyway there's a
whole bunch there for you because it's
basically an API over concurrency that
you know you you really could use yield
from you could use the co-routine
decorators and stuff like that and you
wouldn't necessarily you don't have to
use async IO to do concurrency it's just
it's a very large package to help you do
it so anyways getting back to the code
how do we convert something to being
asynchronous well first of all the main
conversion is rather than def something
you async def something and that
basically notifies python hey this is a
KO routine so the first step that we're
gonna make is actually we're just gonna
a sync both of these functions now the
next thing that we want to do is we want
to go ahead and bring whoops we're gonna
bring in import async I oh that way we
can use some of these like helper helper
functions basically with async IO so now
what we want to go ahead and do is once
you have your ko routines we need a some
sort of loop right right now we're just
running main and in theory main itself
you you would call this at the moment
probably a main loop just because it's
going to
through that little block of code but
what we want to do with async IO is we
need to establish the main loop which is
kind of or if it's basically your event
loop and then you you were gonna add in
tasks and tell that loop how to run and
all that so so now we're going to do is
are going to come down here and this is
where we're gonna stuff in at least the
initial definitions for our loop so
we're gonna say loop equals async IO dot
get underscore event loop so this is
just going to initialize our loop for us
then what we want to do is we basically
want to run the main ko routine until
it's done so the way that we're going to
do that is we're just going to have it's
just its loop dot run run underscore
until underscore complete main and again
there's there's like run
I think it's run forever yeah it's right
here in the documentation yeah you could
run forever you could run until complete
and then there's all kinds of checks
that we can do we can get the status and
all that so again there's no way I can
cover this entire package in one video
or even a few videos just know the docs
exist so so anyways running so complete
and then whenever you're done you can
you do a loop dot close I'm actually not
a hundred percent confident run until
complete is gonna require a loop dot
close but we'll go for it also you'll
see a lot of times in code it's gonna be
like try to do the loop stuff except do
some logging not really a pass and then
finally you're gonna loop dot close
because you always want to close off
that loop this is telling me it's
probably angry don't do anything to okay
exception as he how about that you don't
like that exception no cool general
exceptions okay anyways okay so so now
we've done so we've got the loop now we
have to do is we need to insert stuff
into that loop so all it's gonna do
right now is run the main co-routine
okay but the co-routine is running
basically these functions still sync
right it still says just run one two and
three instead what we want to do is we
want to add them in so rather than find
divisible x' we want to say rather than
just running just that function we
actually want to say loop dot create
underscore task take three there we go
run that so now I'm just gonna add this
bang bang bang bang and now we've
created the task in the loop to actually
run these these three things now I'm
trying to think I think I think this is
runnable at this stage let me just run a
quick see if we hit here yeah okay it is
okay so it is runnable at this stage but
we have a couple of issues the the first
thing is anytime you you send off a
co-routine to run you want to make sure
you wait for the co-routine to finish
especially if you're gonna start a new
loop or if you're gonna end your code so
if you're just if you're writing the
scripts that happens to use async e oh
if you're having anything to do that you
want to make sure you've waited for the
results before you end the script or but
again in a loop especially if you don't
wait you're just gonna you're gonna
really quickly dispatch a lot of code
routines so so what do we need to do we
need to await so we await pretty much
you're gonna await batches or single
co-routines trying to think if I'm gonna
say something wrong here but I'm I'm
trying to think of an instance where you
wouldn't want to eventually await your
Co routine so if you can think of one
let me know I'm sure there's got to be
something but anyways we want especially
if you want the least if you want the
result you're gonna need to await for it
if you're just trying to DA somebody I
guess you don't need to wait for it so
anyways
wait and then we're gonna a sing key oh
I got to stop saying his stinky oh I'm
gonna get stuck saying it now gives one
gives two I see my typos gives three
interested why oh there it is
seize it I did two of them more coffee
please
so now we're going to do is we're gonna
wait on those co-routines to run now if
we run this right now though we're
probably gonna see yeah still runs
everything synchronously and if we were
to let's just make this one order of
magnitude okay two orders of magnitude
coming right up sir
there we go so it's still it's still
hanging right and we're like what's
going on man
so the problem is even though this is a
co routine everything in here though is
still synchronous like there's really no
point where we actually have no IO here
we have no a well okay we have IO but
not not like high latency IO we we're
just running this task so in this case
this is a bulk task this isn't really a
request to a server while we wait for a
response and then during that wait for a
response we could do something else so
instead what I'm gonna do is we're just
gonna add a little tiny bit of weight
basically right here so so in this case
we're just as we iterate through that
range we're asking is it divisible by
what are the div by is if it is we're
gonna append it to located but the other
we're gonna have one more thing we're
gonna say if I modulo and then in this
case we can use anything it depends on
really what this big task is like you
you would probably want to logically do
this you could make it a number as large
as a hundred thousand or larger than a
hundred thousand and then this would
never get hit and I'll show you if
that's not if you're not following my
logic yet that's okay so let me actually
just write it out so let's say 50,000 so
if I is divisible by 50,000 then what
we're gonna do is just momentarily
suspend this this disco routine so we're
gonna oh wait a sinky oh I gotta stop it
was supposed to be a joke and now I
can't stop anyways a sync IO
sleep 0.0004 some
of time again depending on your function
and how many times this might be called
you would probably want to optimize this
in a much better way but this is just
for examples sake okay so all we've done
between running it where this hung and
now it's just added this a little bit so
will this ever run on divs 3 no it will
never hit this as sinky Oh sleep
I think IO sleep will never air on this
one 50,000 it's going to hit it twice
right
and then on this one it's gonna hit it a
bunch so so let's go ahead and let's
just run a sync I oh hey sync yeah so as
you can see it immediately ran instead
our it's interesting it's not doing
we're not finishing 50,000 the 152 we
immediately got our result for the first
one we're kind of waiting on our second
one which is crazy I feel like we should
have had that sleep done by now I don't
know why we're still waiting on that I
should run much much quicker
let me prank it well first of all this
one's huge and if I recall right that's
a thirty-six second I mean just do this
real quick I think that one took 36
seconds to run where's my coffee there's
no three seconds so the other one it's
because of this stupid sleep I'm just
trying to show it for you for examples
sake let's just revert back to 508
thousand and I'll stick with we could
even do five hundred thousand just so
this sleep only gets hit one time ever
so there is going to be a trade-off it's
not like it's the it's super seamless to
like pause and then do some stuff you
don't want to pause like a million times
a second that's not wise but it's still
hanging what am i what do you think
we're doing wrong here
sure why return locate it I feel like if
this something's gotta be wrong so enter
live debug mode so let's just do loop
dot debug or let's see it's I think it's
like loop dot set debug let's do yeah
let's try loop dot set underscore debug
to one see if we can get anything from
this thing one two three one two three
we're waiting yeah like even that
response like that response shouldn't
take that long if I oh my gosh okay okay
whew that's why okay anyways yeah so so
what I bet that's always true we gotta
start again cool alright so what we did
wrong or what I did wrong here is we
never had the double equals zero so
basically it was just what's the
remainder of I divided by 500,000 if
that's not zero congratulations
that's a true statement and then we
sleep so yeah whoops okay back to back
in action now yeah so we should only hit
that one time so hmm what if we made
that 50,000 though you should see that
this yeah okay so so depending on which
way you wanted to run this yes sir which
way you wanted to run this you like so
like this is 50,000 so it's gonna sleep
at fifth the 50,000 mark like on this
one if you use 50,000 it'll sleep twice
right but it'll never sleep on this one
so it's it's fairly reasonable to to see
that when we run this yes it ran largest
middle smallest and then it was like
basically a
lastin first-out kind of situation where
you had and then it solves this one then
this one then this one but if we set
this to be larger than 100,000 so if we
use 500,000 then actually really I'm not
sure why that happened it's a typo
anyways we have 32 10 and then basically
this one then this one gets fed in then
this middle middle one gets solved then
the smaller one gets solved so depending
on which way you wanted your program to
to act you could you could change this
and basically the funk that that there
is that we just never await a asleep and
it's saying an asynchronous sleep so
okay so at this point we basically the
program is asynchronous we're kind of
deciding whether we want to use 50,000
or 500,000 depending on which way we
wanted to run it it doesn't make all
that big of a difference but as we've
seen the sleep itself even if it's a
very very small sleep you're paying a
slight penalty for the trade-off that
you're doing there because again this is
all concurrent this is not parallel if
you're you have to stop and switch tasks
so just like when you stop your run to
tie your shoe that's a little slower
than if you had just kept running right
here you've got bending over time to tie
your shoe right so so just keep that in
mind now I think at this point we've
basically covered everything we we need
to cover but there's a few other little
things actually I think there's one more
essential thing and that is how do we
get the values from our our functions so
first of all just write down here we can
return divs one gives two gives three
now I'll just return them first and then
we'll talk so then once you've returned
these basically when you run this right
here this does return things and it
returns these like result objects so we
could say D 1 D 2 D 3 equals Lu
run into a complete then what we have
are the result objects they are not
results so you actually need to say like
let's say we want to smart print them
out we'll just print D 1 dot results so
we'll just run that result method
basically on that and boom so for D 1
which was here all the numbers divisible
by 4 34,000 113 in that range here they
are so that's how you can get your
results so next what I want to do is
talk about what happens when we make
mistakes so first of all because there's
a few mistakes that you're just simply
gonna make when you're writing
asynchronous programming
I think the mmm there's a view so but
probably the biggest one is is not
awaiting your your your returns so what
if we let me just comment out the
requesting of a result here and then I'm
going to comment out the await and the
return so let's just say we're just
gonna run these these things and in fact
let's leave the await let's run it okay
and we can see that it works
asynchronously it solves the easiest one
first and then the second one and then
the third one what if we comment out the
await so now we're not awaiting anything
what do you think is gonna happen is it
going to run synchronously or is it or
something else right do we have to but
ok so this might be happening because of
debug let me unset the bug yeah so debug
through the airs for us and then even
now since I'm running this in sublime
it's like the equivalent of running it
like in a console if we were to run this
in if we were to run this and in fact it
might even be the loop closed
let me I'm just curious I'm just curious
to this point I actually don't know the
answer to this one if I comment this out
will we get the warning that the task
got destroyed no nice
okay so at this point we can see what's
happened well what has happened is we
started all these co-routines but we can
we can zoom through our script before we
can get the return of the KO routine
because we just dispatched those out and
they're off in the in the they're just
gone now and and and so at this point so
I guess if you don't close your loop so
at least here it would seem to suggest I
just wonder if if this is complete so
maybe that's a reason though why you
should still have loop dot close even if
you use run until complete because what
if you don't complete all right if you
don't complete well you should still
finally close the loop so because if you
do finally close the loop you would see
at least in console form this but if
you're running this like an idle or
something like that you you would never
even see this and this isn't breaking
these aren't breaking errors they're
just they're just like these like
warnings basically it's just telling you
hey something happened but interestingly
enough if you do set debug on and you
run this it actually errors out it gives
you an actual error which i think is
interesting that just setting a debug
will give you errors like that like
it'll break it so anyway it's a useful
thing to use the debug true just because
you're gonna make mistakes like that so
that's one of the mistakes and now what
I'd like to go ahead and do is fix our
code so let's go ahead and bring
everything back okay so the other major
mistake I think is just getting used to
the syntax and if you start writing def
mate it's like what if we if we just
remove a sync from in front of define
main very suddenly you can already see
sublime is telling me but if we've got
to run this boom suddenly we have a
syntax error alright it and at least if
if you're brand new to writing working
with async IO you're gonna probably
forget to do a loop
you're gonna pry forget the async at
least one or two times and then you're
gonna try to use this this sort of logic
here
and then you're gonna be like what was
the syntax again and you're using the
right syntax here you're just not using
the right syntax up here I actually kind
of think I haven't really decided but I
feel like it should probably point back
up to the to the function name or
something or the code what should be a
Co routine name anyway I think that's
enough for a quick intro to async io a
probably do some more stuff on async I
oh maybe maybe do something a little
more project-based than just real basic
like this this is not really super
practical but basically the the the crux
of the situation is it's it's all about
async await and some sort of some sort
of way to suspend things again like I
said before I think it's a io HTTP I
need to open up a search bar a IO HTTP
which I'm pretty sure it's third-party
it is yes it's for async IO but it's an
HTTP client server basically for eight
aged four async IO so really really
common tasks like we were saying before
for why you would want to a
synchronously do things is anything with
web web right and and so if that's your
goal with async io you might want to use
a io HTTP mostly because I think it's
mostly just so you can have the
asynchronous client personally you like
because otherwise you you'd probably
have or at least a session and stuff
like that so working that out and having
everything be really truly asynchronous
would be a lot of work so anyways yeah
check that out so anyway I think that's
all for now if you guys have questions
comments concerns whatever of feel free
to leave them below if you've got
something you want to add did I make a
mistake whatever I'm pretty new to doing
async IO especially in Python so I
probably made a mistake somewhere in
here so anyways that's it for now
otherwise I will see you all in another
tutorial