what's going on everybody and welcome to
part two of our playing with Google
takeout data video miniseries thing in
this video we're going to be doing is
now building some like bar graphs
basically from the top ten we're
probably gonna go with most common words
with a sliding one year window and later
we can try different windows or you guys
can just do it on here on your own time
but for now we'll do a one-year window
so with that let's go ahead and get
started so we should you should at this
point have my life dot database and that
should be populated with you know the
primary key UNIX and then all of the
unique words so what I'm gonna do now is
I'm just gonna copy search database so
I'm just going to rename this to be I
don't know images and then open it sooo
bad I'm gonna close that old one so it
doesn't really matter anymore so now and
in fact I kind of want to keep the
original just so I don't have to retype
some stuff so we still want to connect
to my life database we don't need the
search activity directory anymore we
don't need to parse dates and that
should be good so the next thing I'm
going to do is we want to need a
directory that we're just gonna like
dump all these images into so I'm just
going to make a new directory real quick
I'm gonna call this images let's do word
images and we're gonna start with one
year window one day slide it's kind of a
long plug directory but we'll go with it
that's fine so when I'd well wolf work
that in a minute so now what we're gonna
do is we're just going to specify some
starting constants so add a we always
know is 86400 so 86,400 seconds and then
like a year would be equal to 365 day
and then for our sly
we're going to say that is equal to one
day okay so with that what we're gonna
do is just start generating or word
graph so we're just say define oops
define words graph now what we want to
do is we're gonna save everything into
here so I'm gonna copy that come into
here and I'm gonna say saved or equals
that so that's what we want to stuff all
the images now what we want to do is
first of all we need to starting in an
ending time so we should be able to do
like a query I think so something like I
don't know what it is let's look it up
real quick SQLite max column SQLite max
okay hold on find SQLite find max we
could also try min I just want to see
the expression man
select max from okay so so the first
thing that we could do is we could like
well all we want to do is find the
minimum time and maximum time so the do
words words graph and then we're gonna
say well we're just gonna do a C dot
execute
I guess select max UNIX if you like I'd
like a hair or something over here
select you max UNIX from from words and
then let me just print I'm just gonna
say D equals C dot fetch all and then
we'll print D okay so that's our max
time so then we'll just say max time
equals D D well we can just say this we
can't call to fetch halls but we can we
should be able to just do that and then
0 0 right 0 0 and then we should oh
we're doing this stupid tab thing huh
what why don't you let me do this there
we go thank you sir
thank you that's so generous of you okay
so now max time and then we'll do the
same thing with minimum times boom
change this to be min and then change
this to be min and then we'll print min
time max time run that minimum time okay
cool so that looks like it works pretty
darn well so now we just need to start
sliding over
so our our start time will be equal to
the min time and then our this is like
our current so this is for our window
so our start times min time our n time
for the specific window will be min time
plus I thought we had window what do we
call this thing here
oh we're just calling it here okay I
think I'll call I'll make a variable
called window equals 1 X year cool min
time plus window okay so so that's our
starting our end and then now what we
want to do is we want to just like
iterate over trying to think of how
exactly I would do this so probably I
guess with like a while loop so like Wow
Wow
end is less than max time we want to
make the query for all of the words
between start and end so C dot execute
and we'll make blue tune F string select
word from words oh it looks like we lose
our syntax highlighting when we go F
string on a query that's okay
select word from words where UNIX is
greater than start and unix is less than
end beautiful and then we're gonna say
data data equals c dot fetch all and
then I'm gonna say break here and then
I'm gonna print data fantastic so now
that we have that so this is like all
our individual words in
timeframe now we want to do is take all
of these words and throw them into some
sort of list and then we'll do we'll
create a counter object and then do dot
most common ten so so before I forget
because that's the next thing we're
going to do from collections import
counter done okay
so data so then we want to iterate
through here and append this to words so
while this blah blah blah okay we'll do
words is empty two equal to an empty
list and then what we're gonna say is
four
well it's not rip I guess yeah it is row
and data so for row in data word equals
row zero words dot append row zero now
that's just on the fly but we really
could probably look what could we do
here we could say words equals I zero
right for hmm I in data right we should
be able to get away with that so I am
going to do this
ize ero Oh for okay what do you mean
okay cool alright so that works so that
gives us our words now we can create a
counter object so what we can say is
word underscore counter equals counter
words and then what we could say is we
could say common words equals word
underscore counter dot most underscore
common ten and then rather than printing
all of the words we can print the common
words why did that just happen because
we were printing data I guess let's try
one more time yes so you can see top of
the charts Python not really too
surprising Bitcoin pie probably with
raspberry pie it is kind of weird that
we have more pie than raspberry not
quite sure why that would be maybe I
type owed raspberry that's probably it
probably forgot the P or something but
PI that's fascinating I almost kind of
want to find out if that's right I bet I
just typo'd raspberry but anyway cool
Linux SP for the sp500 again probably
because I don't tell sometimes I type
sp500 sometimes S&P 500 but 500 always
is the case matplotlib I don't know
what's happening here with Google that
would be an interesting one and then
Satan himself so okay so that's our most
common now we'd want to graph those so
that would be on a single time so so how
so we're gonna make a bar graph here and
we need a couple of things when we make
a bar graph I wish bar graphs were
easier in matplotlib which by the way
let's go ahead and import before I
forget let's see so import mat plot mat
plot lib pie plot as put from mat plot
lib import
all styled i use google plot okay great
beautiful couldn't ask for a better set
of imports there so now we need to kind
of set up how we're gonna do this so
common words blah blah blah okay common
words is good we don't need to print
commoners anymore and instead what we
want to say is we're gonna set the y
position so wanna pose equals MP range i
don't think ever apart len common word
some probably ten but later we might add
that to be like a constant that we
change over time let me important numpy
or have I have not import numpy as and P
back to work Y position great and then
we're gonna say word underscore counts
is equal to so actually what we need to
say common words instead what I'm gonna
say is topic 0 for topic in this cool so
it'll be common words and then the word
counts will just kind of be the topic
one so this will be the actual word
itself in the counter because the
counter object was a list of tuples
where the first object in the tuple is
the the word and then the second one was
the count so cool okay so now that we
have that we're ready to actually graph
this and because we're gonna save it we
want to adjust the size right away so
we're gonna say peel T dot figure and
we're gonna say fig size is equal to 12
and seven so that should give us a 1200
by 700 pixel graph so that's our PLT dot
figure then we're going to peel T dot
bar Y paws word counts
and a line underscore or I'm sorry not a
line center and then we're gonna give it
an alpha of nada to pool 0.5 just to
make it semi-transparent love it it'll
be beautiful
and then PL T dot X tix Y pose common
words so that way the X ticks rather
than being like numbers or instead the
actual word cuz that's actually what
we're trying to graph here is the words
and then how much like the y axes will
be volume so with that said PLT de y
label will call this volume and then
finally we'll also do a title and will
display the date in the title so on a
single frame it'll be somewhat legible
but mostly what we want to do is make it
somewhat readable while the frames are
going at like 30 or 60 frames a second
so what I'm gonna do is make this an F
string and I think we're gonna bring it
yet another import from date/time import
date/time and then we'll create a I'm
trying to decide what I want to do there
this is probably the dumbest way to do
it well it's not a dumbest way but I'm
sure someone's gonna come up with a
better way but whatever so what I'm
going to say here is F we're gonna do
dave month year because that'll be the
easiest to follow I think so just
understand that's what's happening all
non-americans are probably very happy
from time stamp so we're gonna get this
date/time object from a time stamp
meaning unix time and we'll go with end
because that's like it's like the last
years of data for today
okay so macro info up to today so we're
gonna use the ending timestamp so this
and then and then what we can say is dot
day month year dot day day there we go
and that gives us our date/time objects
that'll give us a day like
number of the month and then we'll do
month - year so day month and then year
so April 1st 2019 would be one for 2019
and then as these things are changing
really quick we can kind of see it at
least know where we are along the line
so ok so we set the title and then now
we'll run won't just quickly run a PLT
dot show later we're gonna save the
figure rather than show it but okay and
then I guess that's it so let's see it
looks good to me let's go ahead and run
this bad boy that took a long time ok
but here it is hopefully it won't always
take that long ok so here's our graph so
the 14th of June 2014 so we can see here
nice pretty graph now it would be very
tedious to go over this one at a
freaking time so instead what we want to
do is make it into a video so the way
the way we're going to do that is first
we have to save all the images so I'll
probably save all the images and kind of
like pause while I do that and then
we'll pick back up
so let me I'm gonna clear this out of
the way clear this out of the way and
rather than PLT - oh we're gonna PLT dot
save save underscore fit nope underscore
savefig and again to an F string and
it's gonna be saved er right see hmm is
it gonna be saved yeah cuz that's where
we want the images to go and then we're
gonna make the video from the save so
we'll put both in the saved er so saved
her and then and then we want to save
the images with like a counter so we're
gonna start a counter equals 1 alright
so I started at 0 so range is easier
counter 0 and then every time this is
successful we're gonna tick up the
counter Wow
savefig so yeah and then we'll do
counter dot PNG no longer show cuz
that'll take forever and then when all
that is done it's counter plus equals
one and then we would do start plus
equals the window
no plus equals slide right because we
want to slide it by a day and then end
plus equals whew that was rough
okay yes I think we're good I think
we're good
very good let me come down into here
GData and let's go ahead and run images
type ipython Python images PI and let's
see how that goes
oh I wonder if we're printing stuff we
probably don't want to print
we also don't know where we are along
the way let me go ahead and print print
counter I'm gonna save that um I don't
really know if I want to change it let's
just check the images real quick and
just see they're saving open why
everything's going really slow on this
machine why don't you just open an image
bro I have no idea why can't I open this
image there we go
Thanks idiot okay cool yeah so that
looks good I'm just kind of shifting
through the images so now we want to do
is make a video out of those images so
we'll just kind of let this run while we
work on image video so I'm gonna go back
to GData and I don't take images I'm
just gonna copy and paste just have a PI
file basically image video so now we're
gonna do is convert images to V and I've
already got code that does this so I'm
just kind of go copy and paste it so I'm
gonna go to the best Python learning web
site in existence I'm gonna go type in
unconscious or there it is and I'm gonna
go to unconventional neural networks and
then I'm gonna go down to the deep dream
deep dream frames deep dream video is
what I'm after
Thanks and now I'm going to just copy
and pasta all this jazz and then we'll
change what is necessary in a moment
just take note though I do have two
different if you're on Windows use this
if you're on Mac or Linux use this if
you're on Mac I actually don't know out
we do need to modify alt to be the
proper resolution this I'm just gonna
probably hard code so I'm gonna skip
that that's just trying to find the
length of the dream really mm-hmm
out dot release done if you haven't
watched that video by the way check it
out the a deep dream of a neural network
those are actually really cool it's just
like every frame it runs
neural network that kind of like applies
the patterns it sees basically to an
image it's it's super cool definitely
you just have to go see it okay so let's
see how we're doing still going still
making the images good to know good to
know okay so great so now all we're
gonna do is start modifying this stuff
so I don't need dream named Rick budge
what failed to allocate okay that's
gonna be a problem because this takes a
while I really don't want to start that
over again and I also don't know what
that issue is so we should probably end
up making about what 356 times and it
was like five years well a little less
than five years so more than 356 I'd be
more than 1500 so about probably 1800
images so let's go properties dang we're
way off the mark and if we have to keep
doing this every like a few times so
here's what I'm gonna do I'm gonna just
hack it and I'm gonna say import OS and
then we're gonna check to see if this
image exists so we're gonna say if OS
dot path dot is file and then we're
gonna check with an F string for this
like if this image has already been
saved then forget about it let's just
move on
boom what was wrong with you pass else
then we need to create that image so
we'll do all of this up to the counter
tabbed over boom boom
save and let's continue running Python
images dot PI great picked up where we
left off perfect well we'll probably
have to do that like five more times but
hey it's in the name of science
everybody so here let's just say for I
in range for now three hundred it's just
however many images we have says no big
deal image path is going to be basically
this path where the images are right so
we're gonna just copy pasta that into oh
stop hatha joint know this for I this
will just be I dot PNG that's correct
good stuff good stuff saved her we
probably don't have that actually let's
grab that real quick or is it ears
that's kind of gross I don't know if I
want to cuz I'm gonna change this that's
gonna be annoying that's okay though
[Music]
weird I thought I'd deleted these
already
anyway I saved her boom done out again
that'll be this dot and rather than I'll
fix that so this saved ur slash video
dot avi and then it needs to be we're
gonna go with 60 frames a second a
little quicker with 30 frames will take
awhile and we just kind of want to see
it over time so then it would be 1200 by
700 save that and let's just see how
many words we have now our images rather
607 so it'll just say 604 that other
thing to run let me just do this
Python image video pi
that'll create the video hopefully let's
see what happens it looks like it's
doing it that's cool
it'll go in here and it'll save to here
when it's done awesome and let's see
what it looks like so far
I don't care yeah yeah this is cool so I
wish video a V I would go go away so
yeah so here you can actually see is
like certain things like tick up and
then go away so python pandas you got
bootstrap flask and then cut out but
anyways it's kind of cool because you
can like see all this stuff changing
over time 2007 okay we lost Python no
problem we've got some epic error
handling over here failed to allow what
if anybody knows what's going on I don't
think like I don't not sure if we just
ran like a clear fig if that would help
like I'm just not sure failed to
allocate bitmap failed to allocate
bitmap map up
PLT dot close fig but we don't have a
fig we'd have to I guess create a figure
or maybe redefine a figure I don't know
by the time I did bug that will be done
so I'd still really feel like doing it
so now I'm just gonna keep doing this
until we get to like we're done with our
you know probably 1800 ish maybe more
maybe less okay and then when we're done
I'll show you the video and then we can
and then I'll probably just leave it
maybe I'll do another one I just don't
really see much point it's more
interesting looking like I'm interested
in seeing my data and I'm sure some
people are interested in seeing my data
but too bad do it with your own data so
while we wait shout out to some of my
channel members who have been long-term
channel members with me for six and
seven months Joseph Coleman bill fallens
B Luiz Fernando hmm I'm gonna I'm gonna
mess that up
Reginald Roberts and a metanoia guys
thank you so much for your long-term
support you guys are freakin amazing
I wish YouTube gave me better options
for like figuring out when people ivory
upped and even like some of the other
I'm trying to think with you I'm just
like blanking on the name right now but
there's like stream options like people
use for streaming in that whenever
someone really is like not helping
people figure out when someone has like
re-upped so anyways I just have to like
manually go through that list and that's
kind of weird okay so I'm gonna pause
here and then I'll come back when this
stuff is done and we can kind of see at
least my macro search don't think I'm
going to do the micro stuff but if you
wanted to you really would you would
just go over here change it change the
window so this is just gonna be really
big stuff that I did like lots of
searches of and then it's gonna dominate
the stuff that I did you know on a micro
level in them you know if you go down to
the month you'll get they're just much
more granular stuff going on so
definitely really interesting I
encourage you to check it out
with your with your own data anyway I'll
come back when this is done alright and
we are done I went ahead and converted
to video and we can watch in all its
glory so um it was like Python tops the
charts but then you can kind of see are
like you got pandas flasks whatever
Google's do in there you can also see
elite dangerous like elite and then
teachers which is a like a video game
and it's kind of interesting to see how
that kind of goes up pretty high well I
think it made it maybe temporarily to
like fourth and fifth spot and then that
goes away you got seven six one two six
that's a zip code Pokemon comes up pride
for Pokemon go I'm surprised don't so
you go here but whatever then you got
the s2000 pops in climbs like almost top
in the chart really battling Python
they're definitely oh it shoots up and
it wow it definitely passes Python and
OS RS old school runescape shots out Ted
Texas tensorflow
car and Grand Theft Auto towards the end
of 2017
s2000 starts fallin off which is pretty
true I eventually moved away and yeah I
still have the s2000 but I just don't
really drive it very much anymore then
pops up 3-5 404 not yet passing 76126
it's probably take a while like because
we're going back on an entire year when
I actually moved is probably gonna take
like a pretty significant time to
surpass the other one like probably
about six months would be mathematically
it and looking at that's curious anyways
but yeah there's three five four oh four
now on the chart seven six one two six
is gone Porsches coming up old school
runescape top of the chart I don't know
why that happened and then we finish
April 4 2019 old school runescape
killing it it's because I'm a Python
expert I have to Google Python anymore
and three five four oh four zip code
Porsha weather no surprise chaos I'm
surprised Kara's is that high in the
charts to be honest with you
Alabama Tuscaloosa guess where I live
pandas alright
well very interesting okay well
hopefully your data is interesting as
well I strongly recommend you change
some of the windows and then kind of
like look and see how your trends change
over time we could totally easily create
an algorithm that as something like you
would just compare the previous top ten
to the new top ten and anything that's
changed like as one comes out like any
change means one left and one came in or
more it could be two or three but it
would be cool to track that over time
and so you know exactly when those
changes occurred and you could probably
get a pretty like I said because it's a
year like any major event like when I
moved or something like that any major
event would definitely take a decent
period of time to actually shift over so
but I bet I just wonder what would
happen if we went down to like a week or
a month it would be almost perfect like
to figure out when major events change
or like when I bought a new car and like
all that stuff so anyway
pretty interesting and using pretty
rudimentary programming I think we could
find out a lot about people so anyways
cool if you guys want to see more about
playing with Google data let me know
down below what you'd like to see in all
that definitely want to do
text-to-speech with our own voice I
think that would just be really cool so
kind of hoping to do that but we'll see
about that
anyway if you got any other ideas feel
free to leave them below as always
thanks for watching thanks for all the
support subscriptions to memberships the
donations all the fun stuff and I will
see you guys in another video