>> IEL SHIFFMAN: HELLO, EVERYONE! WELCOME TO THIS
LIVE-CODE TRAINING EPISODE, OR EVENT, AND IT IS SPONSORED BY
SPELL. YOU CAN GO TO SPELL.COM/CODING
SPELL.COM/CODINGTRAIN. THERE'S AN INTRO TO SPELL PLATFORM.
WE HAD A TUTORIAL ON HOW TO
TRAIN A STYLE MODEL. WE
HAVE NABIL HASSEIN WHO IS GOING TO TALK ABOUT HOW TO
THANE TRAIN AN LSTM MODEL. THANK YOU TO WHITE COAT
CAPTIONING, THERE SHOULD BE CAPTIONS SAYING THINGS LIKE
MANGO, BLUEBERRY, BANANA, HOPEFULLY. AND I WILL TELL YOU
ABOUT NABIL, HE IS A BROOKLYN-BASED ARTIST, WEB
DEVELOPER, AND ALUMNI OF
NYU. HE .
YOU CAN FIND OUT MORE ABOUT NABIL AND THE WORK HE IS DOING
AND THE PROJECTS HE IS ININVOLVED IN AT THE WEBSITE.
AND NOW, RIGHT NOW, THE DESCRIPTION IS PRETTY BARE, IT
IS NABIL'S BILLOW AND A FEW WORDS ABOUT WHAT WE'RE GOING
TO DO TODAY. SO YOU SHOULD LOOK AT THE LIVE CHAT HERE TO MAKE
SURE THAT EVERYTHING IS WORKING. I WILL BRING NABIL ON AND LOOK
AT THE LIVE CHAT AND LET ME KNOW IF THERE ARE TECHNICAL ISSUES WE
CAN FIX BEFORE WE GET STARTED WITH THE TUTORIAL. ANYTHING I
MISSED? SO I WILL BRING NABIL IN, AND
WE'RE GOING TO AWKWARDLY SWITCH PLACES, I WILL WATCH THE CHAT,
IF THERE ARE QUESTIONS, POST THEM THERE.
OKAY, THANKS. AND HERE WE GO. WELCOME, NABIL.
NABIL HASSEIN: WELCOME, EVERYONE. THANK YOU FOR THE
GREAT INTRODUCTION, AND THANKS SPELL FOR PAYING ME TO MAKE THIS
VIDEO, OR DO THE LIVE STREAM. I HAVE AN OUTLINE OF WHAT I
PLANNED TO GO THROUGH. SO I GUESS I WILL INTRODUCE MYSELF.
I'M NABIL, I LIVE IN BROOKLYN, I'M A FREELANCE TECHNOLOGIST,
EDUCATOR, I DO SOME OTHER THINGS. AGAIN, THANK YOU SPELL
FOR SPONSORING THIS VIDEO. SO THIS LIVE STREAM IS ABOUT HOW
TO TRAIN AN LSTM MODEL USING THE SPELL PLATFORM, SO ON SOME
REMOTE MACHINES SOMEWHERE. AND THEN HOW TO USE THAT MODEL
THAT WE'VE TRAINED USING A LIBRARY CALLED ML5.JS, A
BROWSER-BASED FRONT END LIBRARY FOR USING MACHINE LEARNING
MODELS. SO IN THIS VIDEO, I WILL TRY TO DO A FEW THINGS, TRULY
LIVE FOR YOU HERE TODAY. I'M GOING TO EXTEND A PROJECT THAT I
DID AT THE SCHOOL FOR COMPUTATION, WHICH DAN
MENTIONED, LAST SUMMER. SO THE PROJECT, I CAN SHOW YOU THE
VERSION OF IT THAT IS LIVE ON THE WEB.
SO THIS I TRAINED USING A MARKOV MODEL. SO THIS IS A RHYME
GENERATOR, BASED ON THE LYRICS OF ONE OF MY FAVORITE EMCEES, MF
DOOM. EVERY TIME I CLICK A RHYME BUTTON, IT GENERATES
RANDOM RHYMES PATTERNED ON HIS LYRICS. THIS VIDEO ISN'T REALLY
ABOUT THAT PROJECT, IT IS ABOUT HOW TO USE SPELL. BUT IT GIVE
YOU A LITTLE BIT OF BACKGROUND, AND THE CODE IS LIVE IF YOU WANT
TO -- OR THE CODE IS PUBLIC ON GITHUB IF YOU WANT TO CHECK IT
OUT AT SOME POINT. AND I USED A LIBRARY CALLED
PRONOUNCING, WHICH WAS CREATED -- AND I'M NOT SURE IF TOO MANY
PEOPLE -- A LOT OF WORK WAS DONE BY ALLISON PERISH AND I USED
THAT LIBRARY TO GENERATE A PAIR OF RHYMING WORDS, I TRAINED THE
MODEL BACKWARDS ON THE INPUT TEXT SO I CAN GENERATE THE REST
OF THE SENTENCE GOING FROM THE END TO THE BEGINNING, AND THEN
REVERSE ORDER. DANIEL SHIFFMAN: APPARENTLY
THERE IS STATIC IN THE AUDIO. LET ME SEE IF I CAN HEAR IT
HERE. NABIL HASSEIN TEST : TESTING,
TESTING. SHOULD I MOVE THE MIC? UNCLIP IT FOR
A SECOND.
NABIL HASSEIN: OKAY, SO TALKING ABOUT THE VUD VIDEO, TRAINING AN
LSTM MODEL. SO LET'S GO AHEAD AND GET INTO IT.
SO THE NEXT THING: SO I'M NOT REALLY GOING TO TALK TOO MUCH IN
THIS VIDEO ABOUT THE THEORY OF NEURAL NETWORKS OR WHAT AN LSTM
MODEL IS, BUT IT STANDS FOR LONG-SHORT TERM MEMORY. IT IS A
SPECIFIC TYPE OF RECURRENT NEURAL NETWORK AND WHAT IS
USEFUL ABOUT RECURRENT NEURAL NETWORKS IS THE WAY THEIR
ARCHITECTURE INCLUDES LOOPS. AND THAT CAN BE USEFUL FOR KIND
OF KEEPING DATA AROUND IN THE NETWORKS, SO TO SPEAK, WHICH IS
USEFUL FOR APPLICATIONS INVOLVING NATURAL LANGUAGE,
HUMAN LANGUAGE. BECAUSE CONTEXT MATTERS SO MUCH
IN LANGUAGE. LIKE, PREDICTING THE NEXT
CHARACTER, OR THE NEXT WORD, MIGHT BE -- YOU MIGHT GET A MUCH
BETTER PREDICTION IF YOU ACTUALLY REMEMBER WHAT WAS SAID,
EVEN SOME WHILE AGO, MAYBE MUCH EARLIER IN A LONG SENTENCE. SO
I HAVE A FEW QUICK REFERENCES HERE, WHICH RIGHT NOW ARE A
LITTLE OLD. BUT THIS IS -- THESE ARE WHAT I READ TO LEARN A
LITTLE BIT MORE ABOUT RECURRENT NEURAL NETWORKS. SO THERE'S A
BLOG POST, THE UNREASONABLE EFFECTIVE OVER RECURRENT NEURAL
NETWORKS AND THERE'S A BLOG POST CALLED UNDERSTANDING LSTMS, OR
-- DID I TYPE ALL OF THIS SOMEHOW? I MUST HAVE. I WILL
FIX THAT. YEAH, IT IS CALLED UNDERSTANDING
LSTMS. SO, YEAH, THIS GIVES A LITTLE
BIT OF OVERVIEW OF THE STUFF I WAS TALKING ABOUT. HUMANS ARE
THINKING FROM SCRATCH EVERY SECOND, YOU UNDERSTAND A WORD
BASED ON THE UNDERSTANDING FROM PREVIOUS WORDS, AND WITH THE
NETWORK, THAT'S WHY WE ARE USING THIS LSTM MODEL.
AND, YEAH, I KNOW THAT BEFORE I HAD THE CHANCE -- WHILE
PREPARING FOR THIS VIDEO TO WATCH A VIDEO ON AN OVERVIEW OF
THE SPELL PLATFORM, A LINK TO THAT WILL BE ADDED TO THE VIDEO
DESCRIPTION AND YOU CAN GET MORE IN-DEPTH ABOUT USING SPELL, AND
I WILL MENTION SOME THINGS ABOUT USING SPELL AS WE GO THROUGH
THIS. SO WHEN YOU WANT TO DO A PROJECT LIKE THIS, FIRST YOU
HAVE TO GET YOUR CORPUS OF DATA. AND SO IN THIS CASE, SINCE I WAS
GETTING SONG LYRICS, I USED A SITE CALLED GENIUS.COM, WHICH
YOU MAY BE FAMILIAR WITH, IT A POPULAR LYRICS WEBSITE. AND THE
MAIN THING I USE IT FOR IS READING LYRICS.
AND SO WHAT I'M GOING TO DO, I WILL DO EVERYTHING FROM SCRATCH,
SO TO SPEAK, SO YOU SHOULD BE ABLE TO FOLLOW ALONG IN THEORY.
SO THIS IS OVER A FOLDER THAT I USED TO PREPARE. WHAT I WILL DO
IS MAKE A NEW FOLDER, CALLED SPELL LIVESTREAM, I WILL DO
EVERYTHING FROM INSIDE OF THIS FOLDER, WHICH IS LIVED SOMEWHERE
IN MY COMPUTER. SO RIGHT NOW, THIS FOLDER IS EMPTY.
AND SO THE FIRST THING THAT I'M GOING TO DO IS CLONE MY
GENERATIVE DOOM REPOSITORY FROM GITHUB. AND THERE IS ONLY ONE
FILE IN THERE THAT I CARE ABOUT, SO MAYBE -- MAYBE I WILL NOT
CLONE THE WHOLE FILE STORY. I WILL GET THE ONE FILE. SO I'M
JUST GOING TO -- I DON'T KNOW WHERE THIS IS.
DID I PUSH IT UP? I HAVE SO MANY BRANCHES HERE. I WILL USE THE
ONE THAT I HAVE ON MY COMPUTER. SO I'M JUST GOING TO COPY A FILE
THAT I HAVE ON MY COMPUTER INTO THIS FOLDER. SO WHERE WAS THAT,
IN SPELL
SPELLDEMO/GENERATIVEDOOM/DATA. I HAVE A FILE, INPUT.TXT THAT I
BROUGHT A COPY OF INTO THE CURRENT DIRECTORY. SO YOU CAN
SEE THIS IS
JUST A LIST OF LYRICS, THIS IS MY CORPUS. AND IT IS WORTH
NOTING THAT THE DATA SET I'M USING FOR THIS EXAMPLE ISN'T
THAT BIG. WE CAN CHECK THE SIZE OF IT WITH THE COMMAND LINE
UTILITY, DU, FOR DISK USAGE, AND YOU CAN TELL HOW BIG THIS FILE
IS. IT IS ABOUT 308 KILOBYTES. IT IS NOT HUGE.
NORMALLY WHEN YOU ARE TRAINING MACHINE LEARNING MODELS, THE
MORE DATA, THE BETTER. BUT I GOT ALL THE LYRICS I COULD FIND.
THIS IS REALLY THE MOST THAT I COULD GET. SO THAT'S WHAT WE'RE
GOING TO USE. COOL.
SO IT IS ALSO WORTH NOTING THAT YOU HAVE TO CLEAN THE DATA
BEFORE YOU TRAIN IT. SO I CAN ACTUALLY GO AHEAD AND SHOW -- I
CAN GO AHEAD AND SHOW THE CODE THAT I USED TO GET THE LYRICS.
IT IS ON MY GITHUB IF YOU WANT TO CHECK IT OUT.
SO LET'S PUT IT
OVER HERE. SO I HAPPENED TO DO MY SCRAPING
USING PYTHON, YOU CAN USE ANY LANGUAGE, NODE.JS, RUBY, OR
WHATEVER YOUR LANGUAGE IS. I USED A PYTHON LIBRARY, BEAUTIFUL
SOUP, WHICH IS VERY USEFUL FOR WEB SCRAPING. AND SO IT SO
HAPPENS THAT GENIUS.COM HAPPENS TO KEEP THEIR LYRICS IN, LIKE,
THEIR URLS, THEY FOLLOW A PATTERN LIKE THIS,
GENIUS.COM/ARTIST NAME AND THE NAME OF THE SONG, AND THEN I
USED ANOTHER PYTHON LIBRARY, REQUESTS, TO JUST GO AHEAD AND
FETCH ALL OF THESE DIFFERENT THINGS. SO THIS IS THE BASIC
IDEA, I'M NOT GOING TO GO INTO FULL DEPTH, BUT I HARD-CODED A
LOT OF NAMES OF DIFFERENT SONGS INTO HERE AND THEN I HAVE A MAIN
LOOP WHICH LOOPS THROUGH EACH ARTIST'S NAME, BECAUSE DOOM HAS
RECORDED UNDER DIFFERENT NAMES. AND THEN THE SAME THING FOR THE
ALBUMS, AND THEN THE SONGS, IN ORDER TO GO AHEAD AND FETCH ALL
OF THIS
DATA. WHEN YOU GO DIRECTLY TO SOME,
YOU KNOW, LYRIC'S WEBSITE, WHEN YOU FETCH THE DATA ON THE PAGE,
YOU GET A LOT OF OTHER DATA YOU DON'T CARE ABOUT IN THE HTML.
SO YOU HAVE -- AN IMPORTANT STEP IS TO CLEAN THE DATA, SO WHEN
YOU ARE TRAINING THE MODEL, YOU ARE ONLY TRAINING IT ON THE
CORPUS THAT YOU CARE ABOUT, YOU ARE NOT TRAINING IT ON THE
BRACKETS OF THE HTML TAGS THAT YOU DON'T ACTUALLY WANT.
SO I THINK I HAVE THE CODE THAT I USED TO CLEAN IT ON GITHUB,
AND IF NOT, THERE ARE OTHER RESOURCES YOU CAN USE TO LEARN
ABOUT DATA CLEANING. THIS IS ABOUT TRAINING MACHINE LEARNING
MODELS USING SPELL AND USING THEM IN THE BROWSER. LET'S GO
BACK TO THAT. AND PROJECT GUTENBERG, THAT'S A RESOURCE
WITH A LOT OF FREE TEXT IN THE PUBLIC DOMAIN THAT YOU CAN JUST
USE. AND WEB SCRAPING WITH NODE.JS IS ANOTHER RESOURCE FOR
DOING THIS KIND OF THING. AND ALTHOUGH MY SCRAPE.PY FILE IN
THE DOOM REPOSITORY DOESN'T SHOW THIS, THE ORIGINAL VERSION, I
JUST KIND OF KEPT EACH FILE -- I KEPT EACH SONG IN ITS OWN FILE
OF LYRICS. BUT THE MACHINE, WHAT I'M GOING TO SHOW YOU NEXT,
WORKS WITH AN INPUT THAT'S ONE BIG INPUT FILE, INPUT.TXT AND I
DID BORING SHELL SCRIPT STUFF THAT CONCATENATES THE FILE
TOGETHER. AND THAT'S ALL I WANTED TO SAY ABOUT GETTING
DATA. SO LET'S GET INTO THE MAIN THING. SO I DID THIS PART,
I CREATED A NEW DIRECTORY FOR ALL OF THIS STUFF TO
LIVE IN. SO IT IS A GOOD HABIT TO USE
VIRTUAL N, AT LEAST WITH PYTHON DO, I UNDERSTAND THAT THINGS
HAVE MOVED ON A BIT WITH PYTHON THREE, BUT I'M ON THE PYTHON
TWO. SO I'M GOING TO USE
THIS VIRTWOMAN UAL N THING TO KEEP MY DEPENDENCIES ISOLATED.
AND I HAVE SOME OTHER VIRTUAL ENDS ACTIVE, I SEE IT FROM MY
COMMAND PROMPT OVER HERE. I RAN THE COMMAND, DEACTIVATE, TO GET
RID OF THAT. I WILL CLEAR THE SCREEN, TO MAKE IT LESS NOISY
HERE. AND AND THEN I WILL CREATE A NEW VIRTUAL N, SPELL
VIDEO
VIRTUAL N. SO IT IS SETTING UP A NEW
VIRTUAL ENVIRONMENT WHICH LIVES INSIDE
OF THIS FOLDER HERE. AND THE WAY YOU USE PYTHON VIRTUAL END
TO GET IT ACTIVE, SPELL,
VIDEO VIRTUALENV, SOURCE, AND THEN THE PATH TO
ACTIVATE SCRIPT. AND NOW YOU CAN SEE MY PROMPT
CHANGED BECAUSE I HAD MY TERMINAL PREFERENCES SET UP THAT
WAY, I CAN REMEMBER WHICH VIRTUALENV I'M IN.
OKAY, SO I DID THAT. I GOT THE INPUT FILE, WHICH I SHOULD PUSH
IT UP. I HAVEN'T ACTUALLY PUSHED UP TO THE GITHUB, LIKE
THE ONE FILE VERSION, BUT LIFE IS MESSY.
SO I WILL GET THE PROPER VERSION FROM GITHUB IS THIS REPOSITORY
CALLED TRAINING LSTM. SO I'M GOING TO CLONE THAT. AND WE
WILL GO AND TAKE A LOOK AT THAT REPOSITORY AND ITS READ
ME. . I WILL CLONE, COPY, AND PASTE
IT IN. SO YOU CAN SEE THAT THIS -- FROM
THE DESCRIPTION OF THIS REPOSITORY, TRAINING IN LSTM
NETWORK AND USING THE NETWORK ML5.JS IS HIGHLY RELEVANT TO
WHAT WE ARE DOING IN THIS VIDEO. SO THIS REPOSITORY'S READ ME ARE
BASED ON WHAT YOU WOULD WANT TO DO IF YOU WERE TRAINING THE
MACHINE LEARNING MODEL LOCALLY ON YOUR OWN COMPUTER.
AND SO IF THAT'S WHAT YOU WANT TO DO, YOU CAN GO AHEAD AND
FOLLOW THIS. SINCE THIS VIDEO IS ABOUT HOW TO USE SPELL,
THAT'S WHAT CAN ARE I WILL DO. I WILL NOT FOLLOW THE DIRECTIONS
EXACTLY, BUT CERTAIN PARTS OF IT. SO I CLONED THIS
REPOSITORY, THAT'S THE LAST COMMAND THAT WHERE RAN. I RAN.
I WILL ENTER INTO THAT REPOSITORY. AND THEN WHAT I
WANT TO DO IS CREATE A DIRECTORY INSIDE OF HERE CALLED DATA. AND
THENP I'M GOING TO GO AHEAD AND MOVE AHEAD THAT INPUT.TXT FILE
INTO THAT DATA DIRECTORY. OR A COPY OF IT, RATHER. I
COULD DELETE AND GET A DIRECTORY, BUT WHATEVER.
OKAY, GREAT. SO THIS IS THE SET-UP.
WE HAVE A REPOSITORY, WE HAVE THIS REPOSITORY LOCALLY THAT IS
GOING TO HELP US TRAIN THE LSTM NETWORK USING TENSORFLOW. AFTER
WE TRAINED THE MODEL, WE CAN USE IT IN ML5.JS. SO WE'RE DONE
WITH THE SET UP, SO LET'S GET INTO TRAINING THE MODEL.
SO AGAIN, THIS IS THE LINK THAT YOU CAN USE TO SIGN UP FOR SPELL
IF YOU HAVEN'T ALREADY. IT SO HAPPENS THAT I HAVE ALREADY.
SO I SHOULD BE LOGGED IN HERE, I WILL MAKE SURE I DON'T HAVE TO
LOG OUT, I HAVEN'T. SO I AM IN HERE IN SPELL.RUN AND
IT GIVES ME INFORMATION ABOUT HOW TO INSTALL SPELL, HOW TO LOG
IN WITH SPELL, THERE'S A QUICKSTART GUIDE AND THE
RESOURCES THAT I USED IN PREPARING FOR THIS VIDEO.
SO LIKE I MENTIONED, THE OTHER TRAINING LSTM REPOSITORY SHOWS
YOU HOW TO RUN LOCALLY. FOR US, ALL WE NEED TO INSTALL IS SPELL.
SO I WILL GO AHEAD AND DO THAT. I WILL PIP INSTALL SPELL AND IT
WILL FETCH SPELL AND WHATEVER THINGS THAT SPELL DEPENDS ON
FROM THE PYTHON PACKAGE, PYPY, OR WHATEVER IT IS CALLED. IT
WILL GO AHEAD AND GET THAT. AND THEN ONCE IT IS DONE
INSTALLING, I WILL BE ABLE TO
LOG IN, I WILL USE MY USER NAME AND PASSWORD IF I CAN REMEMBER
IT. I CAN REMEMBER IT. YOU CAN SEE
THE READ ME, THAT IS ME, I'M LOGGED IN AS MYSELF.
AND IF YOU EVER FORGET THAT YOU ARE LOGGED IN, FOR SOME REASON,
THIS SPELL COMMAND HAS A LOT OF SUB COMMANDS. SPELL WHO AM I
WILL TELL YOU WHO YOU ARE. BUT I'M GOING TO GO AHEAD AND GET
STARTED WITH TRAINING THIS MODEL. AND THE FIRST THING THAT
WE NEED TO DO IS TO UPLOAD THE FILE TO SPELL.
OKAY? SO WHAT I WANT TO RUN IS THIS
COMMAND HERE, SPELL UPLOAD THE PATH OF THE LOCAL COMPUTER ON TO
THE FILE. AND THEN I WANT TO GIVE IT A NAME OF WHERE I'M
GOING TO UPLOAD IT TO. SO SPELL UPLOAD MY
DATA/INPUT.TXT TO THIS DEST
DESTINATION. IT WILL WANT IT MOMENTARILY. SO I WILL TELL
SPELL TO UPLOAD THE FILE I WANTED TO UPLOAD. AND THEN IT
TELLS ME THAT THE FILE, THIS IS THE PATH TO IT ON MY LOCAL THAT
I'M TYPING ON RIGHT NOW, IS ACCESSIBLE AT
UPLOAD/NAME/INPUT.TXT. AND THAT'S THE NAME THAT I WANT TO
PUT IN. SO I'M GOING TO DELETE THE PART I DON'T WANT AND PUT
IN, WHAT WAS
IT, NABIL SPELL LIVESTREAM DOOM. OKAY?
I WILL UPLOAD AT 307K, THAT IS CLOSE TO WHAT WE SAW WHEN WE RAN
THE DU COMMAND EARLIER. AND THE UPLOAD IS COMPLETED.
SO THAT'S GREAT. AND SO WHAT WE'RE GOING TO DO
NOW, AND THIS IS KIND OF THE MOST -- PROBABLY THE MOST
COMPLICATED COMMAND WE ARE GOING TO RUN, BUT THE ONLY ONE. AND
THIS IS WHAT IT IS SAYING TO ACTUALLY GO AHEAD AND RUN THE
PYTHON SCRIPT THAT WE DOWNLOADED FROM TRAIN, THAT TRAINING LSTM
GIT REPOSITORY. OKAY?
WE'RE GOING TO RUN THAT, THIS TRAINING PY SCRIPT, WITH THE
DATA SETDATA SET DEAREST TO WHAT I CALLED DATA, THAT'S THE NAME
OF THE FOLDER WHERE I PUT THE INPUT.TXT. AND I'M GOING TO RUN
IT, AND I'M GOING TO MOUNT THE DATA -- I WILL MOUNT THE FOLDER
THAT I CREATED BY UPLOADING THAT FILE.
THAT IS AS THE NAME DATA SO IT CAN UNDERSTAND THIS DATA
DIRECTORY. SO I SHOULD GET, I THINK, ONE
ERROR WHEN I DO THIS AND I WILL TALK ABOUT WHY.
SO SPELL RUN, MOUNT, UPLOAD, NABIL, I CALLED IT
LIVE STREAM. -- THIS IS FROM WHEN I
PRACTICED. I WILL GO AHEAD AND
FIX THAT. IT TELLS ME THAT THERE ARE
UNTRACKED FILES ON THIS REPOSITORY AND THEY ARE NOT
AVAILABLE ON THIS RUN. THAT'S THE FILE THAT I CARE ABOUT.
SO SPELL ENCOURAGES US TO USE THE GIT VERSION CONTROL SYSTEM
TO MAKE SURE THAT THE DATA THAT WE'RE TRAINING ON IS CHECKED IN
AND THAT IS VERY GOOD FOR
REPRODUCEANT ABILITY IF WE WANT TO UNDERSTAND WHAT WAS THERE.
SO I WILL DO AND TALL LOW FOLLOW THE SUGGESTED
WORKFLOW. SO I WILL GIT, ADD DOT, OR WHATEVER IN THIS
PARTICULAR CASE. AND I WILL SAY ADD DATA.INPUT.TXT OF DOOM
LYRICS. AND NOW, HAVING DONE THAT, IF
I RUN THE SAME COMMAND, IT WILL NOT GIVE ME THE SAME WARNING
SINCE THE FILES ARE TRACKED INSTEAD OF BEING
UNTRACKED. YEAH, SO LET ME GO AHEAD AND
SEARCH THROUGH THIS, AND I WILL MENTION THE OTHER THING I SAID I
WAS GOING TO MENTION. I WILL PRESS UP A FEW TIMES AND RUN THE
SAME COMMAND AGAIN. SO RUN, I'M MOUNTING THE DATA FOLDER I
UPLOADED TO BE CALLED DATA. AND I PUT IT IN THE
WRONG FOLDER. DANIEL SHIFFMAN: IF YOU ARE
GOING TO MOUNT THE DATA FOLDERDO , DO YOU HAVE TO COMMIT IT? SO
IF YOU DON'T WANT TO UPLOAD IT WITH GIT, THEN YOU CAN DO --.
NABIL HASSEIN: OH, RIGHT. SO THERE IS MORE THAN ONE WAY TO
WORK WITH SPELL, AND I CONFLATED THE TWO OF THEM A LITTLE BIT.
SO, YEAH, I DIDN'T ACTUALLY DO THE GIT COMMIT BECAUSE OF THE
WAY THAT I'M DOING -- THAT I'M DOING THIS, BECAUSE I UPLOADED
IT BEFORE. AND THEN I WILL ALSO GET THE SAME REPRODUCIBILITY
BENEFITS, BECAUSE SPELL WILL KEEP A RECORD OF WHAT WE
UPLOADED, BUT IT DOES NOT HURT TO GIT COMMIT IT EITHER. SO I
WILL FIX THAT TYPO. DANIEL SHIFFMAN: IN THIS CASE IT
IS A SMALL FILE, IT DOESN'T MATTER, BUT IF IT IS HUGE, YOU
WANT TO UPLOAD THE SEPARATELY WITHOUT HAVING TO COMMIT IT.
NABIL HASSEIN: YES, GIT IS NOT THE BEST FOR COMMITTING LARGE
FILES, SO THERE ARE TOOLS LIKE SPELL AND GIT, THERE'S A BUNCH
OF TOOLS. SO COOL. SO LET ME GO AHEAD AND FIX THAT
TYPO AND IN MY NOTES AS WELL. IT TELLS ME EVERYTHING IS UP TO
DATE, BECAUSE I DID MAKE A COMMIT, AND LIKE I MENTIONED, I
DOESN'T HAVE TO. IT IS TELLING ME IT IS CASTING SPELL IN 16, I
HAVE USED IT ABOUT 15 TIMES
BEFORE. I CAN REQUEST AND IT IS RUNNING, AND IT IS STARTING TO
TRAIN THE MACHINE LEARNING MODEL AND GIVING ME SOME DATA ABOUT
HOW THAT TRAINING IS GOING IN THESE, YOU KNOW, IN THESE
DIFFERENT POINTS. DANIEL SHIFFMAN: 30 MINUTES HAS
PASSED. NABIL HASSEIN: AND I'M BACK.
AND THIS IS STILL RUNNING, AND LIKE IT TOLD ME BEFORE, I CAN
JUST GET OUT OF THE LOGS IN MY LOCAL TERMINAL, CONTROL CC.
THIS IS RUNNING ON A REMOTE MACHINE THAT IS PROVISIONED FOR
ME. I'M NOT WORRIED ABOUT IT. SO I'M NOT STOPPING THE RUN FROM
HAVING MY CONTROL C, JUST FROM THE LOGS FROM APPEARINGING IN MY
OWN TERMINAL. IF I WANT TO CHECK OUT THE LOGS, I CAN SAY
SPELL LOGS 16, AND THEY WILL START APPEARING AGAIN.
AND THERE ARE SOME OTHER COMMANDS IT TOLD ME ABOUT. I
CAN KILL IT WITH SPELL KILL, I WILL NOT, BECAUSE I WANT IT TO
KEEP RUNNING. AND BESIDES CHECKING OUT THE LOGS LOCALLY
WITH SPELL LOGS, THE NUMBER OF THE RUN, YOU CAN ALSO COME OVER
HERE TO THIS SPELL -- IN THIS SPELL WEB UI AND CHECK OUT
DIFFERENT INFORMATION ABOUT THE RUN IN HERE. BUT UNTHUS VIDEO I
HAVE A PREFERENCE FOR THE COMMAND LINE. I WILL DO IT THAT
WAY. SO COOL, LET'S SEE. AND ONE
THING I WANTED TO MENTION IS THE PERIMETERS IS CALLED
HYPERPERIMETERS OF THE NETWORK. SO LET'S GO BACK TO THE GIT READ
ME. SO THIS GIVES YOU INFORMATION
ABOUT HOW YOU WOULD RUN IT LOCALLY, AND YOU CAN PASS
ADDITIONAL FLAGS TO CONTROL MORE OF THE CHARACTERISTICS OF THE
NETWORK: ITS SIZE, HOW MANY LAYERS THEY ARE, THE LENGTH OF
THE SEQUENCE, AND VARIOUS OTHER THINGS THAT YOU CAN READ MORE
ABOUT IN THIS REPOSITORY. THEY HAVE SOME RECOMMENDATIONS FOR
WHAT YOU MIGHT WANT TO SELECT FOR YOUR HYPERPERIMETERS,
ACCORDING TO THE SIZE OF YOUR TRAINING DATASET. BECAUSE MY
TRAINING DATA SET IS SO SMALL, I DECIDED TO
DEFAULT. THOSE ARE THE OLD NOTES, I DID
THAT PART. OH, YEAH.
AND THE NEXT THING THAT I WANTED TO TALK ABOUT IS THE
DIFFERENCE BETWEEN RUNNING ON CPU VERSUS GPU.
AND I'M A FAN OF REVIEWING MATERIAL. CPU IS THE CENTRAL
PROCESSING UNIT OF YOUR COM COMPUTER WITH A LITTLE BIT OF
PARALLELISM, BUT FOR THE MOST PART IS BEST AT RUNNING THINGS
SEQUENTIALLY REALLY FAST. AND THE MODEL OF COMPUTATION IS
DIFFERENT FROM A GPU, AT SOME POINT IT STOOD FOR GRAPHICS
PROCESSING UNIT, AND MAYBE IT STILL DOES, BUT THAT ACRONYM IS
RETIRED BY NOW BECAUSE GPU HAS APPLICATIONS OTHER THAN
GRAPHICS, INCLUDING TRAINING NEURAL NETWORKS ON TEXT. GPUS
HISTORICALLY, THEY GOT THAT NAME BECAUSE, EACH
PIXEL ON A SCREEN IS INDEPENDENT OF EACH OTHER ONE AND CAN BE
COMPUTED INDEPENDENTLY. AND A GPU IS MUCH MORE HIGHLY PARALLEL
COMPARED TO A CPU. IT IS NOT GOING TO BE AS FAST AS
COMPLETING ONE SINGLE TASK, BUT IT -- IT IS VERY GOOD FOR
DISPLAYING THINGS ON SCREENS AND ALSO VERY GOOD FOR TRAINING
NEURAL NETWORKS. SO IN THE LAST COMMAND THAT I
RAN OVER HERE ON THE COMMAND LINE TO TRAIN THE NEURAL
NETWORK, THIS IS RUNNING VIA A CPU.
AND WHAT I CAN DO, IF I WANT SPELL TO RUN MY CODE ON A GPU, I
WILL TELL IT THE BEST TYPE OF MACHINE I WANT BY ADDING THIS
DASH DASH MACHINE TYPE FLAG. AND THEN THE MACHINE TYPE I WILL
USE IS K80. WHERE DOES IT COME FROM?
IF YOU CHECK OUT SPELL.RUN/PRICING, YOU WILL SEE
INFORMATION ABOUT HOW MUCH SPELL CHARGES FOR YOUR MACHINE TYPES,
WHETHER THIS CPU OR THAT GPU IS BEST FOR THE PARTICULAR TASK.
THERE'S CPU, CPU BIG, KED -- THAT IS ONE OF THE LESS
EXPENSIVE GPU UNITS. AND THAT'S WHAT I'M GOING TO
USE. EVERYTHING IS UP TO DATE, WE SEE
A VERY SIMILAR BIT OF OUTPUT AS WE DID BEFORE AS IT IS READY TO
START TRAINING THE
MODEL. IN A MOMENT WE SHOULD START
SEEING THE LOGS, I'M SURE IT READS THE TEXT
FILE. I WILL LOOK AT THE NOTES WHILE I
WAIT
FOR THAT. SO THAT'S COOL, ALL RIGHT.
SO THIS RUN IS RUNNING, AND I DON'T KNOW HOW OBVIOUS THIS IS
IF YOU ARE FOLLOWING ALONG. THIS IS HAPPENING A LOT FASTER,
THIS MODEL IS BEING TRAINED A LOT FASTER BY A GPU THAN THE CPU
ONE WAS. AND SO THE CPU ONE GOT A HEAD
START, BUT I STILL EXPECTED THE GPU ONE WILL FINISH
SUBSTANTIALLY FASTER. WE ARE AT 113 OUT OF 6,000 ITERATIONS THAT
IT IS GOING THROUGH IN SHARING THE MODEL. IF WE CHECK IN ON
THE PREVIOUS ONE, I WILL SEE HOW FAR IT IS.
THE HEAD START I HAD WAS PRETTY BIG. BUT YOU CAN SEE THE GPU
ONE IS MOVING FASTER. AND LIKE IF WE ACTUALLY GO IN,
BECAUSE LIKE IT I MENTIONED BEFORE, I HAD A FEW PRACTICE
RUNS HERE BEFORE. WE CAN LOOK AT A FEW ONES THAT I DID BEFORE.
AND USING A SLIGHTLY DIFFERENT MODEL, WE CAN SEE HERE.
BUT THIS ONE, ON CPU, TOOK CLOSE TO FIVE YEARS AND ON GPU IT TOOK
ONLY MORE THAN 15 MINUTES. AND, YEAH.
SO GPU IS FASTER FOR THIS PARTICULAR USE CASE.
SO JUST FOR THE
SAKE OF TIME, I WILL GRAB A MODEL THAT I TRAIN ED BEFORE.
SO WHAT I WANT TO DO TO GRAB THE DATA FROM SPELL IS TO RUN
THIS COMMAND HERE. THIS IS SPELL CP RUNS/THE NUMBER
OF THE RUN/MODELS. THAT'S HOW I WILL FETCH THAT DATA.
SO I'M JUST GOING TO SEE IT UP HERE AND NOW I'M IN MY HOME
FOLDER OF ALL THE DIFFERENT THINGS THAT I'M GRABBING FROM
HERE, GRABBING FROM THERE, TO PUT TOGETHER INTO THIS DEMO. I
WILL GO AHEAD AND RUN SPELL CP, AND I WILL LOOK HERE AGAIN.
SO YOU CAN SEE THAT IS USING THE SAME TRAINING, LSTM, THAT I WAS
TALKING ABOUT. IT COMPLETED IN 5 MINUTES, THIS SHOULD COMPLETE
PRETTY QUICKLY, TOO. AND THAT WAS THE NEXT RUN THAT I DID JUST
A FEW MINUTES BEFORE THE LIVE STREAM STARTED. SO SPELL CP
RUNS/15/MODELS. AND IT IS COPYING 10 FILES THAT
ARE FROM
THERE. SO THIS IS LS DATA, IT REMEMBERS
THE SAME DATA DIRECTORY THAT I PASSED IN BEFORE AS THE NAME.
AND THESE 10 FILES CONSTITUTE THE MODEL.
OKAY? AND I'M NOT GOING TO GO INTO
DEPTH ABOUT WHAT ARE THESE FILES, WHAT IS IN THEM.
BUT IF I FOLLOW ALONG, YOU CAN BUILD INTO THEM AND CHECK IT
OUT. COOL.
SO WE FRAMED THE MODEL. WE HAVE USED SPELL TO TRAIN AN LSTM
MODEL ON A CORPUS OF DATA THAT WE OBTAINED, AND NOW THAT WE
HAVE THE MODEL, LET'S USE IT FOR SOMETHING.
SO I'M GOING TO BORROW AND THEN MODIFY AN EXAMPLE FROM THIS
REPOSITORY HERE, ON THE ML5.JS GITHUB ACCOUNT, THEY HAVE A
REPOSITORY CALLED ML5
EXAMPLES. SO THERE'S A COLLECTION OF
EXAMPLES, THERE'S A BUNCH OF THEM, YOU CAN FIND ABOUT HOW TO
ORGANIZE AND SOME OTHER STUFF FROM THEIR READ ME.
AND I WILL USE ONE IN THE P5.JS FOLDER, WE ARE WORRIED ABOUT
LSTMS, AND THIS INTERACTIVE ONE IS INTERESTING. THERE'S MORE, I
MEAN, IT IS -- I'M NOT GOING TO DESCRIBE IT. WE WILL USE THE
NON-INTERACTIVE VERSION, THE LSTM TEXT. AND WE HAVE HERE
JUST A FEW FILES, SO THEY ACTUALLY HAVE A PRE-TRAINED
MODEL THAT I'M GOING TO IGNORE AND NOT USE BECAUSE WE'RE GOING
TO USE OUR MODEL THAT WE JUST TRAINED. AND WHAT I'M GOING TO
DO IS JUST FETCH THESE TWO FILES, THIS HTML FILE AND
SKETCH.JS FILE AND BECAUSE THIS REPOSITORY IS BIG AND I JUST
DON'T WANT TO WAIT TO CLONE IT, I'M GOING TO FETCH THESE TWO
FILES AND NOTHING
ELSE. SO WHAT I'M GOING X2 TO DO IS
JUST CREATE A DIRECTORY, AND WHAT NOT. AND THIS IS ML5 LSTM
EXAMPLE, AND I WILL CHANGE THE CURRENT DIRECTORY TO BE IN
THERE. I WILL CLEAR THE SCREEN FOR CLARITY'S SAKE. I WILL USE
THE COMMAND LINE PROGRAM, W GET, WHICH WILL FETCH THE RAW FILE,
AS A QUICK RAW IN GITHUB, AND WILL FETCH IT INTO MY -- ON TO
MY LOCAL MACHINE. I WILL DO THAT. AND THEN I GO
BACK AND I DO THE SAME THING WITH SKETCH.JS. I WILL FIND THE
ONE RAW FILE, COPY THE URL, AND I WILL USE THE PROGRAM W GET TO
DOWNLOAD IT LOCALLY. SO NOW IF I LIST WHAT IS HERE, I
WILL HAVE THESE TWO FILES, INDEX.HTML AND SKETCH.JS AND WE
WILL TAKE A MOMENT TO CHECK OUT -- WE WILL READ THE FILES
THEMSELVES AND USE THEM. SO WHAT I'M GOING TO DO IS RUN A
PROGRAM, HTTP SERVER, WHICH YOU CAN INSTALL IF YOU WANT. IF YOU
DON'T HAVE IT, YOU CAN RUN IT WITH NPM INSTALL, AND IF YOU ARE
USED TO USING A DIFFERENT WEB SERVER, ANYTHING THAT WILL SERVE
UP AN INDEX.HTML IN YOUR LOCAL FOLDER IS FINE. SO IT TELLS ME
WHERE, THE URL THAT I CAN GO TO ON LOCAL HOST TO CHECK IT OUT.
SO I'M GOING TO GO THERE. AND SO LSTM TEXT GENERATION
EXAMPLE, THIS IS A CORPUS ON VIRGINIA WOLF, I'M NOT GOING TO
DO THAT, SO I WILL CHANGE THAT. SO I WILL GO INTO THIS FILE AND
ALSO LOOK AT THE
JAVASCRIPT FILE. SO I WILL OPEN UP THE REPO,
SPELL LIVE
STREAM. THESE ARE THE TWO FILES THAT I
DOWNLOADED A MOMENT AGO, INDEX.HTML AND SKETCH.JS. I
WILL OPEN IT UP IN A WAY THAT IS MORE READABLE FOR
YOU. SO I WILL DO IT OVER HERE, THEN.
THE FOLDERS DO NOT REALLY BOTHER THE VIDEO AS MUCH AS THIS ONE
DOES. OKAY, SO WE HAVE HERE AN HTML
DOCUMENT WHICH RELIES ON P5 AND ML5 AS THE LIBRARIES THAT ARE
BEING USED. AND PRETTY MUCH NOTHING ELSE.
SO THIS EXAMPLE USES A PRE-TRAINED MODEL ON A CORPUS OF
MF DOOM. I WILL MAKE IT
NICE MEANING OF LIFE IS -- THIS IS SOMETHING THAT I DON'T
REMEMBER DOOM SAYING, WE CAN REMEMBER IT AS THE SEED TEXT FOR
NOW. IT IS INPUT.HTML, I CAN CHANGE IT. WE HAVE A FEW
SLIDERS OF HOW LONG THE OUTPUT WILL BE AND THE TEMPERATURE,
WHICH I WILL TALK ABOUT LATER. AND WHAT IS INTERESTING IS THE
SKETCH.JS FILE, WE WILL TAKE A
LOOK THERE. OKAY, SO WE DECLARE A FEW
VARIABLES HERE. THIS VIDEO IS NOT ABOUT P5, BUT IT IS A COOL
LIBRARY, I ENCOURAGE YOU TO CHECK IT OUT IF YOU ARE NOT
FAMILIAR WITH IT ALREADY. IT IS GOOD FOR ARTISTIC CODING AND
OTHER PROJECTS AS WELL. SO THERE ARE TWO
MAIN FUNCTIONS IN -- I'M NOT GOING TO GET INTO P5.
SO WE'RE GOING TO START WITH THE SET-UP FUNCTION, AND P5 WILL RUN
THIS SET-UP FUNCTION FOR US IN THE BEGINNING. AND IT SAYS
CREATE THE LSTM GENERATOR, PASSING THE MODEL DIRECTORY. SO
I DON'T HAVE ANYTHING CALLED MODEL/WOLF BECAUSE I DID NOT
CLONE THE WHOLE REPOSITORY. I NEED TO MOCK MAKE SURE THAT
THIS, WHEN IT IS GENERATING, LIKE WHEN IT IS CREATING THE
LSTM THAT WE'RE GOING TO USE, I NEED TO MAKE SURE THAT THIS IS
POINTING TOWARDS THE PROPER PATH WHERE WE HAVE OUR FILES. SO LET
ME GO AHEAD AND REMIND MYSELF ON THE COMMAND LINE OF WHERE I'M
KEEPING EVERYTHING. I WILL START THE SERVER AGAIN IN
A MINUTE. SO THIS APPEARS Y HAVE DATA UP
THERE, WHICH IS WHAT I WANTED. AND SO I WILL COPY THAT FOLDER
INTO HERE WHERE I AM. I WILL SAY WHAT IS IT, LOWER
CASE R, CAPITAL, FOR RECURSIVE COPY.
AND THIS IS HERE, BESIDES THE TWO FILES THAT I FETCH FROM
GITHUB USING W GET, I ALSO HAVE ALL OF THE DATA FILES RIGHT
HERE. AND SO WHAT I'M GOING TO DO IS
CHANGE THIS TO SAY DATA, THAT'S WHERE MY DATA IS.
AND THEN HERE, THERE IS OTHER CODE, THIS IS ABOUT THE USER
INTERACTION OF WHAT HAPPENS WHEN SLIDERS AND GET MOVED AROUND AND
BUTTONS ARE CLICKED, SO I'M NOT GOING TO GO OVER THAT.
AND WHAT WE WILL JUST FAKE -- TAKE A MINUTE TO LOOK AT IS THIS
GENERATE FUNCTION. AND AGAIN, WE'RE NOT GOING TO GO ALL THE
WAY THROUGH, BUT IT LETS YOU KNOW IT IS GENERATING STUFF,
JUST SO THAT THE USER KNOWS SOMETHING IS HAPPENING, IT GRABS
THE INPUT FROM THE SEED TEXT, AND IT USES THESE PERIMETERS:
TEMPERATURE AND LENGTH. THE ML5.JS LIBRARY DOES THE HEAVY
LIFTING FOR US. WE CALL THE
LSTM GENERATE FUNCTION WITH OUR DATA AND PATCH UP THE CALL-BACK
FUNCTION WHICH WE WILL UPDATE THE DOM AND THE HTML PAGE WHEN
THAT FUNCTION IS DONE RUNNING, OR DONE GENERATING THE TEXT THAT
THE MODEL HAS PREDICTED BASED ON THE INPUT SEED.
OKAY? SO YOU CAN SEE THIS IS A PRETTY
SHORT FILE, AND THEN WE WILL GO THROUGH EVERY DETAIL, BUT IT IS
ON GITHUB, YOU CAN CHECK IT OUT. I MADE ONE SMALL CHANGE TO IT.
SO I WILL GO BACK TO MY NOTES, I AM PRETTY SURE I KNOW WHAT I
WANT TO DO NEXT BUT IT IS GOOD
TO BE SURE. I WILL RUN THE SERVER AGAIN, IT
IS UPDATED WITH THIS. THE MODEL IS
LOADED. CLICK GENERATE, AND I DON'T KNOW
HOW MANY PEOPLE LISTEN TO DOOM, BUT THIS IS SOMETHING THAT HE
MIGHTSY. WE CAN ADJUST THE LENGTH TO MAKE IT LONGER OR
SHORTER, AND THEN WE CAN USE THIS TEMPERATURE THING, LIKE THE
TEMPERATURE IS SOMETHING LIKE INTUITIVELY, IT IS THE
RANDOMNESS OF THE TEXT. THE HIGHER THE TEMPERATURE, THE LESS
RANDOM IT WILL BE, THE MORE DERIVATIVE IT WILL BE OF THE
ORIGINAL TEXT. AND IF YOU TURN IT UP VERY HIGH, IT IS LIKELY
THAT YOU GET DIRECT QUOTES FROM THE ORIGINAL CORPUS U. IF IT IS
LOWER, THEN MAYBE IT IS A LITTLE BIT MORE CHAOTIC, SO TO SPEAK.
AND THAT CAN GENERATE THINGS THAT ARE A LITTLE BIT MORE OUT
THERE OR ORIGINAL. IF YOU START TO DO IT TOO LOW, YOU GET
NONSENSE. THEY WILL NOT MAKE MUCH SENSE
AT ALL. I WILL HOLD MY OPINION ON THE
GENERATIVE LYRICS FOR NOW. THAT'S REALLY THE MAIN THING.
IF I WAS GOING TO GO AHEAD AND REPRODUCE MY ORIGINAL PROJECT,
WHAT I WOULD DO NOW IS PULL IN ANOTHER DEPENDENCY WHICH IS THE
PRONOUNCING LIBRARY AND THEN I WOULD HAVE, WHAT I SHOULD HAVE
DONE, I DO THIS NOW, IS TRAIN THE MODEL
BACKWARDS. IF WE LOOK AT THIS INPUT.TXT, YOU CAN SEE THESE
LYRICS ARE FORWARDED AND THE MODEL DOES IT ALSO. THAT'S
SOMETHING I WANT TO DO IS TO REVERT THE INPUT, TRAIN THE
MODEL BACKWARDS, I CAN USE THE PRONOUNCING THING TO GO
BACKWARDS. INSTEADS OF DOING THAT, IT WOULD MAKE SENSE TO
TAKE QUESTIONS FROM PEOPLE ON THE LIVE STREAM, BECAUSE THAT IS
PRETTY MUCH IT. SO WE GOT THROUGH WHAT IS LSTM,
GETTINGGETTING DATA, SETTING THINGS UP, TRAINING THE MODEL
USING SPELL IN ML5.JS. SO I LOOK FORWARD TO ANY QUESTIONS.
DANIEL SHIFFMAN: I WILL SLIDE NEXT TO YOU, I MIGHT HAVE SOME
QUESTIONS. SO I LOVED THAT CREATIVE
SOLUTION, I NEVER THOUGHT OF THIS, THAT I THINK IT IS
INTERESTING TO REALIZE THAT THE TEXT, THE NEURAL NETWORK ISN'T
LEARNING ANYTHING ABOUT THE MEANING OF TEXT. YOU THINK, OH,
MACHINE LEARNING, AI IS DOING WHAT A HUMAN BEING DOES, BUT IT
IS LEARNING ABOUT THE SEQUENCE OF WHAT CHARACTERS TEND TO
APPEAR AFTER ANOTHER. SO WE CAN JUST AS EASILY LEARN IT
BACKWARDS OR FORWARDS. NABIL HASSEIN: AND ESPECIALLY
WITH THE MARKOV MODEL THAT I USED, THE MARKOV PROPERTY MEANS
THERE IS NO MEMORY. SO IT WAS A VERY NATURAL THING TO DO. BUT
IT WAS A LITTLE WEIRD. DANIEL SHIFFMAN: ONE OF THE
QUESTIONS THAT CAME UP, SOMEONE NOTICED -- I WILL MAKE SURE THAT
THE MIC IS ON. YOUR FILE SIZE WHICH IS 300KB
WHICH IS PERFECT FOR A
QUICK DEMONSTRATION LIKE THIS. DOES IT GIVE YOU A SENSE OF HOW
MUCH DATA THAT YOU NEED TO GET THE RESULTS YOU MIGHT WANT TO
GET? NABIL HASSEIN: MORE DATA IS
BETTER, IT DEPENDS ON THE APPLICATION. I WANTED THE
LYRICS TO SOUND LIKE THIS PERSON. AND I MEAN, YEAH, 3KB
OF LYRICS IS A GOOD AMOUNT FOR
AN ARTIST. AND I WOULD SAY GENERALLY MORE
IS BETTER, AND IT REALLY DEPENDS ON THE APPLICATION.
DANIEL SHIFFMAN: AND IT DEPENDS ON WHAT YOU ARE
TROIG TRYING TO GENERATE. IF IT IS TWO LINES, THAT IS EASIER TO
GENERATE THAN A NOVEL. AND I HAVE TUTORIALS
ON MARKOV MODELS, AND WITH LARGER.WE ARE GOING TO PRODUCE
MORE SURPRAISES
PRIZES AND
CREATIVE OUTPUT. AND USE SPELL CP ON A FINISHED
RUN, IS THAT THE SAME AS A FRESH RUN? WHAT HAPPENS IF I USE SPELL
CP WHEN IT IS TRAINING? DANIEL SHIFFMAN: WE SHOULD CHECK
TO SEE IF IT FINISHED, AND I THINK THE ANSWER TO THIS IS THAT
SCRIPT, THAT PYTHON SCRIPT SCRIPT THAT WE'RE USING DOES NOT
GENERATE THE FINISHED MODEL FILES UNTIL IT IS DONE. SO
THERE IS NOT ANYTHING TO SPELL CP, BUT MOST TRAINING SCRIPTS
WILL GENERATE IN PROGRESS VERSIONS, CHECK POINTS, A
VERSION OF THE MODEL IN PROGRESS, AND DAYS AND DAYS AND
DAYS, IT CRASHES IN THE LAST 10 SECONDS, YOU HAVE SOMETHING THAT
YOU CAN WORK WITH. THE BASIC EXAMPLE FROM THE ML5 PROJECT
WHICH IS BASED ON TENSORFLOW -- I DON'T THINK WE ARE GENERATING
THE CHECK POINTS, MAYBE WE ARE, WE SHOULD. SO WE WILL ADD THAT
PROJECT. NABIL HASSEIN: BOTH RUNS
FINISHED. THE GPU RUN WAS 3 TIMES
FASTER, IT TOOK 5 MINUTES, VERSUS 17 ON CPU.
DANIEL SHIFFMAN: AND BECAUSE IT IS SUCH A SMALL AMOUNT OF DATA,
THAT DIFFERENCE IS LESS EXTREME. IF YOU HAD 10MB OF TEXT OR 100MB
OF TEXT AND YOU ARE WORKING WITH AN IMAGE DID I TELL --
DATABASE, IT CAN BE DAU
AYS OR WEEKS LESS. NABIL HASSEIN: AND YOU CAN
START THE RUN, DO WHATEVER YOU ARE DOING, AND THEN THE MODEL
FINISHES AND YOU CAN USE IT LATER INSTEAD OF LEAVING THE
COMPUTER RUNNING, RUNNING ON CPU INSTEAD OF GPU BECAUSE YOUR
LAPTOP IS NOT AS POWERFUL AS THE DATA CENTER MACHINES.
DANIEL SHIFFMAN: AND THEN YOU CAN GO TO SLEEP AND WAIT
INTEREST THE
-- FOR THE NOTIFICATION TO COME IN.
AND TRUNCATED SENTENCES, I KNOW WHAT IT DOES, GENERATE 100
CHARACTERS. IT IS ARBITRARY OF WHERE IT CUTS OFF. IN THE
PREVIOUS VERSION OF THE PRODUCT, WAS THAT A CONSIDERATION, WHERE
TO END IT? NABIL HASSEIN: THERE'S A FEW
THINGS. THESE LSTM MODELS CAN BE TRAINED BY CHARACTER OR BY
WORD. IF YOU TRAIN BY WORD, YOU SHOULD BE GETTING WORDS. THAT'S
ONE THING YOU CAN DO. I DID LEAVE SOMETHING, AND I HAD
SOMETHING ELSE. WHAT WAS IT? THIS IS THE PYTHON VERSION
THAT I
HAVE. DANIEL SHIFFMAN: AND YOU CAN
FIND A NICER POINT TO END. IF YOU ARE TRAINING OFF OF SONG
LYRICS, THE LINE BREAK IS A CHARACTER THAT THE MODEL WILL
LEARN. BUT IT WILL GENERATE THINGS AND PUT THE LINE BREAKS
IN THEM. SO IT DEPENDS ON HOW YOU FORMAT
THE TRAINING DATA
. NABIL HASSEIN: I CUT IT OFF AT
THE SPACE. DANIEL SHIFFMAN: HOW DO YOU
USE SPELL WITH JAVASCRIPT? SO IN THIS LIVE STREAM TODAY, WHAT
NABIL SHOWED USH US IS A PYTHON SCRIPT. WE SWITCHED TO
JAVASCRIPT TO EXECUTE THE MODEL IN THE BROWSER TO MAKE IT
INTERACTIVE, BUT THE TRAINING WAS IN PYTHON AND THAT IS
TYPICAL, MOST MACHINE LEARNING TRAINING SYSTEMS AND THINGS YOU
WILL FIND ARE IN PYTHON OR SOMETHING LIKE TORCH IS A THING,
AND THERE ARE OTHER SCIENTIFIC COMPUTING LANGUAGES AND
PLATFORMS FOR DOING MACHINE LEARNING.
HOWEVER, YOU MIGHT WANT TO RUN A NODE SCRIPT, NODE IS A
JAVASCRIPT FRAMEWORK FOR DOING SERVER-SIDE PROGRAMMING AND
OTHER STUFF, TOO. AND SO THERE IS ACTUALLY TENSORFLOW HAS --
JAVASCRIPT TENSORFLOW RUNS IN NODE. SO WHAT THE SPELL FOLKS
MEANS IS THAT THERE'S AN ARGUMENT WHICH IS DASH DASH, YOU
CAN FIND IT IN THE DOCUMENTATION SOMEWHERE, DASH DASH APT, AND
THAT'S WHERE YOU CAN TELL IT -- WE DID NOT HAVE TO INSTALL
TENSORFLOW AS IT DEPENDENCY, BECAUSE I THINK SPELL INCLUDES
THOSE. IN THE LIVE STREAM FROM BEFORE SHE HAD TO ADD AN
ADDITIONAL DEPENDENCY. YOU CAN CONFIGURE THE SPELL COMMAND TO
SAY DASH DASH APD SPACE NODE AND IT KNOWS I NEED TO USE NODE FOR
THIS. I WILL PUT A VIDEO TO SHOW YOU WHERE TO RUN THE
DOCUMENTATION. NABIL HASSEIN: YOU CAN TELL
SPELL TO RUN ECHO --. DANIEL SHIFFMAN: ANYTHING YOU
CAN TYPE INTO THE CONSOLE OR TERMINAL IN YOUR COMPUTER, YOU
CAN RUN INSTEAD OF ON YOUR COMPUTER ON THE SPELL PLATFORM.
AND
THIS IS WHAT HAPPENS
ON THIS LIVE STREAM, IT IS SUPER INTERESTING AND WE ARE JUST
TALKING ABOUT THE TEXT EDITOR. IT IS NOT SOMETHING THAT I -- I
DID NOT LEARN BY, I
LEARNED PROGRAMMING -- I WAS FAR ON THE OTHER SPECTRUM INTERFACE.
IT IS GREAT TO SEE THAT, YOU ARE WORKING WITH HOW EFFICIENT IT
IS. NABIL HASSEIN: YES, IT TAKES AN
INVESTMENT, TO BE HONEST, I DON'T KNOW IF I RECOMMEND THAT
OTHER PEOPLE START USING IT. IF YOU HAVE TIME TO LEARN IT, IT IS
NICE THAT IT INTEGRATES NICELY WITH PYTHON, JAVASCRIPT, AND A
MILLION OTHER LANGUAGES. AND I KEEP MOST OF MY NOTES USING THE
FORMAT, WHICH LETS ME DO THIS NICE OUTLINE STUFF AND STUFF
LIKE THAT. DANIEL SHIFFMAN: OKAY, I THINK
-- I WAS GOING TO BE LIKE, OH, WE'RE DONE. THERE IS ONE MORE
QUESTION. CAN THE MODEL COPY THE EXACT LYRICS IN THE DATASET?
INTERESTING QUESTION. NABIL HASSEIN: IT CAN, I DON'T
KNOW IF WE WANT IT TO. THE MORE -- THE HIGHER THE TEMPERATURE
IS, THE MORE LIKELY IT WOULD BE. DANIEL SHIFFMAN: THE POINT HERE,
YOU HAVE A DIFFERENT CONCEPTUAL IDEA FOR WHY YOU WANT TO USE
THIS MODEL. AND WHAT NABIL IS DEMONSTRATING IS THE IDEA OF
HAVING THE MODEL GENERATE SOMETHING THAT IS SIMILAR IN
STYLE AND SUBSTANCE, BUT NOT THE SAME. AND CERTAINLY,
PROBABILITY-WISE, THERE IS LIKELIHOOD THAT IT IS THE EXACT
LINE FOR THE LYRICS. THERE ARE SO MANY COMPUTATIONS AND
PERMUTATIONS THAT IT IS NOT LIKELY TO HAPPEN.
SO I THINK WE'RE GOING TO WRAP UP. THIS WAS AMAZINGLY
EFFICIENT, A ONE-HOUR LIVE STREAM, IT NEVER HAPPENS, AND WE
ONLY HAD A FEW GLITCHES. AND ESPECIALLY BECAUSE OF THE
AUDIO PROBLEMS WE HAD EARLIER, WE WILL HAVE AN
EDITED VERSION OF THIS. AND THIS ARCHIVE WILL BE
AVAILABLE AFTER I TURN IT OFF. YOU CAN WATCH IT, AND I WILL GET
THE NOTES FROM NABIL AND PUT EVERYTHING IN THE DESCRIPTION.
IF YOU WANT THE COMMANDS AND THE URLS, AND A LOT OF PEOPLE CAN
SEE IN THE CHAT, THE IDEA OF A RECURRENT NEURAL NETWORK IS A
NEW CONCEPT, SO THE READINGS ARE REFERENCES TO GET AN
UNDERSTANDING OF WHAT THE PIECES ARE. SO STAY TUNED FOR THAT,
YOU CAN ASK QUESTIONS LATER IN THE COMMENTS AND I WILL ALSO BE
BACK THIS AFTERNOON BECAUSE I'M DOING A SERIES ON MAKING
MASTADON BOTS. I WANT TO SHOW HOW TO RUN A PROSING SKETCH TO
GENERATE. SO LOOK FOR THAT AROUND FOR THAT AT 3:30PM, FOR
AN HOUR OR SO. THANK YOU SO MUCH. AWESOME FOR
YOU TO BE HERE, GREAT TO SEE ANOTHER EXAMPLE OF TRAINING
WITH PYTHON. SO I WAS SAYING THIS EARLIER, I
DON'T HAVE ANY GOOD SYSTEM FOR THIS. I WILL GO TO THE WEB
PAGE, THE CAMERA WENT OFF. PERFECT TIMING. I WILL TURN IT
ON SO WE CAN WAVE GOOD BYE AND I WILL HIT STOP STREAMING AND WE
ARE DONE FOR THE DAY. IF YOU WANT TO TRY THIS, YOU CAN
GO TO
SPELL.RUN/CODINGTRAIN TO SIGN UP FOR AN