as your project begins to grow the conversation of whether or not you want to keep everything
in one project or actually break it out into multiple repos could come up and there's a
lot of discussion on the dbt discourse and the community about how you should go about
that the most common one is obviously the one repository but what we're going to talk
about here is how you could set it up to have separate repositories for teams or focus areas
along with one shared repository and we'll do that by referencing multiple git packages
so really what we're looking at here is how we can have one shared repo maybe with some common macros
or models and then reference that in some of your other projects but all within you know your same
organization i'm going to show just a very simple example of this it's going to just have one macro
but the concepts are the same i have this one demo dbt project that i've been working on
throughout most of these videos but i also have this other kind of random project that i have
that i'm going to use as an example to say i want to reference a macro that i add in this project to
my other one and the way we're going to do that is like this if we look at the packages documentation
rather than adding a package from dbt hub we're going to add one directly from another git url
just to give you a quick overview let's say this dbt music data project is the
shared repo and this demo dbt project is one of these other ones we want to be able to reference
something from this shared one in our case dbt music data on here demo dbt what i've done is i
just created a macro on here i just called it new macro it's very simple it's literally just saying
insert a value hello name and print it out obviously in a real scenario you're going to
have many macros with some way more complicated logic but the process is still the same
so let's go and show you how this works as it says here we want to add a packages.yaml
and reference it by the git url i'll copy this here and i've already created a packages.yaml here
on my demo dbt i'm going to paste this in here and here's where we put the url of what we want to
reference and in this case i want this dbt music data so i'll go ahead here grab this copy this url
and replace it here now revision this is where you want to put a tag or a branch name if you have one
here it says they strongly recommend that because if you think about it if you just put it to
master let's say without any specific release version or anything like that you make a change to
master and somebody comes in here and reruns dbt depths you could have introduced something that
could break it so it's very important that you're specific in this scenario i do not have a release
so what we'll just do is kind of obviously against what they are just saying unfortunately but
i did want to point that out but for this case we'll just say master for the sake of example
so now again we're running this in demo dbt packages.yaml we're trying to bring in the project
of our other repo which is this dbt music data so i'm going to run dbt depths and it should bring
it in now we can see here it is giving us that warning it's pinned to the master branch which
could introduce changes you know that's understood for this example and now if we go to dbt modules
we see music data and we can see everything here so at this point we've now brought in this project
we showed here before that there was an actual macro here that we wanted
and it's actually not yet in the master branch and i did that on purpose to show how this part works
although i don't have a release number i do have a branch so if we go to add macro branch on here
again we're looking at this music data one here is where i've added this macro so what we'll do
is change this to add macro so that means it's going to look here for this branch specifically
and again if you had a release number you could put that here so now let's rerun dbt depths
so here it is it's now in here in our dbt modules under music data and we can use
this we can now use this in our project so let's walk through how to do that
i'll go into this example teams model here and i'm just going to add this as
a column it doesn't really matter this is a macro so normally you would call this macro
by names in this case we've called it new macro with a name and normally this would be it if this
was something that was in your folder in your project under macros of your actual project
of your you know root project here this would work but let's see what happens when we do a dbt
compile we're getting an error and it says new macro is undefined and that's because
it's looking in this project that we're working in which is demo dbt it's looking under macros
and saying there's nothing called new macro but we do have it here and the way to call this to
indicate that this is the one that we want we have to prefix this with the name of the project
music data dot new macro so that way it knows look at this here under macros and here's what we want
so let's try this again and there it worked and we should be able to see this under the target
compiled and here it is hello mike and that was from what we put in here so this is critical here
make sure that you're adding the identifier for the specific project and if we look back here you
know if maybe you have multiple repos depending on how you set this up you'll have multiple projects
that come in here ultimately a package is a project and in order to identify the specific item
or node that you want you need to make sure you identify the project as well
last thing i'll point out here is if you do not already have a get ignore set
you will want to add this i think it comes default on a lot of projects now but if you don't make
sure that you have amongst these other things dbt modules included here so that way anything in here
that you bring in is not going to impact your changes and be brought into some code reviews
because really that's not important for this particular project here that you're working on
again very simple example but the idea is the same that as you want to bring in
other projects maybe in your organization your different repos that you're working with
this is the way that you can do it and this is how you reference it