in this video we're going to take a look at a really cool feature of github which is github
actions and how you can add them to your project
the idea of continuous integration and deployment is becoming an increasingly more important part
especially for data engineers and actions allow you to have workflows to automate that process
so here we are in the project and i've gone over to the actions tab and here it gives you a lot
of information at a high level understand that actions are what's going to allow you to automate
certain scripts testing deployment things like that and all of this is handled through github
workflow files what we're going to do is just click set up this workflow and what this is
going to do is within your project within the github directory.github if you already have
it it's putting it within a new workflows directory and it's just called blank.yaml
obviously that's not what you want to keep we'll change this to something else
in this case let's call it dot yaml this can be a little overwhelming at first
when you see it it might seem super confusing especially if you're not used to yaml syntax
or linux scripting or any of that stuff but i promise you once you get the hang of it you start
to really pick up what's happening and it's just a matter of kind of finding and replacing things
so let's walk through the syntax briefly on what we're even looking at
from the top here this is saying on so when will this action run when is this workflow going to
be triggered the default here is saying on a push to this branch on a pull request against main and
these are things that you can edit as much as or a little as you want it's totally up to you they're
just putting some defaults in here down here it's also saying on workflow dispatch meaning you can
run it manually and we'll see what that looks like a little bit later a workflow is made up of
one or more jobs this right here is the job so the top here is saying when will we run this and this
is saying what will we run so we have a one job here called build what will this run on ubuntu
latest and this might be a little confusing as well if you're not used to linux this is a linux
operating system and essentially what's gonna happen here is behind the scenes github through
this feature of actions is going to spin up a virtual machine or a container i think it's a
container that is using this operating system you can change this to something else i think you can
choose windows or mac and everything that follows is operating on that machine so you can almost
think about it it's as though you've went to aws or azure and went in and made a virtual machine
and tried to run something but instead this is doing it for you and the idea is that every time
this runs it's going to be a fresh build so that you're sure that what you're testing is consistent
every single time and then the steps are the next part uses actions.checkout slash checkout
we'll touch on this in a minute this is referring to kind of a pre-packaged set of things that are
going to happen and all this is saying is it's going to check out and essentially use the files
in the directories of whatever branch you're currently on and then we have these next two
ones which are just simple run commands and run as it says is going to use the runner's shell and in
our case we're using a linux shell so this would be bash so that you can assume that this would
be the equivalent of opening up a bash terminal and running something echo hello world here you
can run a couple grips in a row and to simplify i'm going to take out all these comments when you
get rid of all those comments it's actually only a couple lines and that's something we can work with
so i'm going to start a commit let's just get this built onto our main branch and then we can see it
actually run commit new file so now this is there and remember we did say anything push to master
will trigger it so if we go to actions now here we can see it's running this workflow on the push
to master this was the name of the commit that we had we see ci that's the name of the workflow
that we gave it if we go in here there's only one job that job build and here are each of the steps
by default there are a couple of things that happens it sets up the job and completes then
we have this run actions this is that checkout action we talked about it's gonna run a one
line script just that hello world again just from that linux bash command this comes built in with
a post run that is going to close this out so all this stuff now set up automatically for us to run
before we get into actually editing this a little bit i want to show you some of the documentation
and where that lives and touch on some things that we were looking at before so i've opened up a few
links here here's the docs for github actions there's a lot of information here honestly in my
opinion learning github actions is not something that you can just read and you'll just understand
it you really have to play around with it and run some of your own tests and examples get the
hang of what's happening and see your way through it so a popular link here is the workflow syntax
and this again this is a lot of information at first and it can seem intimidating
but really it's more of a reference so as you start to run into different scenarios
and you're like okay how do i do this type of action or this command you can look through
here and read through it and try to find what you need for example this on command we said on push
to branches main you know for example we don't have to specify the branch they're saying you can
you can add a list you can exclude certain things and ignore certain branches or tags it's really
customizable and all this stuff is here if you need it the other item i want to touch on real
quick is this uses actions checkout i remember when i started this was really confusing what
does this even mean what does what are we talking about here and what this is is it's referring to
this right here github.com actions these are publicly available actions that you can use
for example we're using checkout v2 and if you were to look up checkout action for checking
out a repo this is a very popular one so really what you're doing is inheriting a project and
using some of its files here we can see usage and within here it'll give you some more specific on
how you can do different things with it customize it how you want maybe you only want to check out
a specific branch every time you know up to you but all that information is available here
let's go into our project here i'm going to do a git pull so that i now have this new workflow
file in here so first what i'm going to do is i'm going to edit a little bit of this so instead of
working with branch main here let's just do let's delete this i'm going to do on push to test ci i'm
going to do that because what i'm also going to do is hit checkout test ci so now if we look at
our branch this is what we're doing and what i'm saying here is only run this when there's a push
to this branch we'll also have the ability to do it manually if we want i'll just keep that there
let's add another step here for example let's just say we want to add run another script and the name
is what will come up in our workflow and we'll just say echo this is a new command just get this
pushed through and push this to our remote branch and now if we go back to actions we can see this
automatically picked it up because within the file it's saying look out for this scenario
and that scenario happened so now if we go in here it succeeded so it ran all these things
and we can see this new step that we added run another script this is a new command and it ran
successfully now what i want to do is take it another step forward and start to incorporate
of that third party github action that we can reference
and the scenario that i'd like to play out here is we're going to create a python script that creates
a new file but in order to do that we need to first install python onto our virtual machine you
got to think of it as that virtual machine that's spinning up it's a blank slate so we need to make
sure we specifically say everything that we want to put on it so let's look for python here we see
set up python usage see it's similar to the checkout what we'll do is we'll add a step that's
to install python i'm going to put this side by side here so we can see i'm going to get rid of
these guys here let's do a few things first and let's see if start to follow along here i
know i'm moving a little quick but hopefully this will make sense let's first let's list all files
and instead of an echo we'll just run ls so imagine it's going to run ls in the bash
command in the context of this repo that's where that's where you are so we would expect it to list
all these files and we'd only expect to see these two so i'm actually going to also create
a new python file called createfile.p and really what i want to do is so we're going to list all
files create a new file and then list the files again but first we also need to install python
so install python create a new file and then list the files again to prove that python worked
so to create this file i'm going to just copy and paste this in here it's essentially just a quick
script to create a new file but let's rename this and call this something more interesting
so this file is available for us obviously we haven't checked it in yet what we want to do is
list the files so we'll actually see these three first next let's add a step to install python and
what we'll do is copy this right here indentation right and what this is saying is the name of the
step is this we're going to use this project this action and one of the options we have here is with
and we can identify the version we'll do 3.8 so it's saying install python version 3.8 we're going
to list the files to all python 3.8 and then let's run a python script you can use this command with
the next step run python myscript just like you would locally instead of my script we're going to
do create file.py and we would expect that file to create a new file at the end list the files
again so we can see the new files we would expect this new one to now show up we'll say list all
original files install python run the python script and then list the files to confirm they're
there and hopefully that made sense and again all of this is going to happen on this virtual
machine here this runner machine which is ubuntu latest and this is only ever going to happen
when we push this branch which we're about to do shortly let's go ahead and add this now git
push let's go to actions now and now we can see it's running again and if we go into here let's
go to build so here that ran very quickly if we go to the list we can see these files the readme and
create file now we've installed python onto this runner it's going to run that script we can see
kind of where that's happening so by that point it would have created that new file because that's
what the script did it ran that script in that directory now let's list the new ones and here
we can see it now built that new file for us in that same directory we don't see it here because
obviously we didn't run that script here but we ran it on the runner machine and here it is for
us and now it'll just run this post action and close it out for us so like i said that is just
a brief intro on what you can expect how github actions work and as more projects move towards
version control and being in code it gives you more flexibility to automate so many different
things i hope this was helpful if you did find it helpful feel free to like subscribe if you want to
see more videos we'll upload every week and i'll see you in the next video thanks for watching