Welcome everybody you're watching mr fugu data science. Today we want
to look at duplicate data frame column names. I'm going to show you
what we're dealing with, what you should avoid and a solution of how i fixed it.
Let's get into this it's a short video here's my socials feel free to hit me up. All the code will
be in the link in description below as well as in the comments section. Both areas whatever suits you.
I have affiliate links you can feel free to check out as well as different ways
of helping support the channel. Such as super thanks, memberships or buy me a coffee. Let's go.
Since this is a short video i'm going to code this out for you okay. We need Pandas numpy. I
think we need numpy and collections. Let's look at what the data are because i can't remember.
Oh it's some sales data that we're dealing with. Let's look at the column name since those are of
utmost importance today. So this particular column is one that was a issue for me doing some database
work. So i renamed it with the outermost portion of the json nesting and did an underscore for the
portion that was held with inside of it to help with the naming issues. Since we know that this
is the problem we're dealing with. We're going to take this and artificially recreate it so it's
like this again so we have the duplicates so i can show you what's up. First, if you're doing this and
you weren't sure or you were wanting to look for duplicate values. You would have to do something
similar to what i'm going to show you now to find the location. If you wanted to get a specific
column location you would run something like this. We know that our problem child is in number 40 now.
I want to change this by setting this variable to rename our column where column that we're
trying to update the name for position 40. And we want to name it as the same as before. We're number
40 let's rename, and let's get our column. And here we are. Now it's artificially updated. Now let's
play around with this and fix it. The online solution would tell us to do this. We'd be told
to rename this column. Let's rename this column. We're going to take this and modify it slightly.
So let's take the updated portion that we had before and see if this works.
And then we're going to print this off so we know that this may work or may not work.
Oh oh no. Oh no. What happened? Our position two changed and was updated ah drats. A/nd now we
got this one. Well that's kind of pointless what just happened? We were updating the column names
but because they were the same here we had an issue and it updates both. Well what's the
workaround I could do real quick to fix this? Because this sounds like it's kind of useless.
Here was my work around to fix this. Well let's do a list comprehension real quick
and we're going to count the number of occurrences that we have for our variables using the counter
from collections. And we're taking all the column names of parse and we're counting it if anything
is greater than one. And we need to do dot values here and store this as a list
items, not values. And we find that this count is doubled. We already know this but this is
a self-check for you if you're in the future trying to find out how to evaluate this for your
data. Now we're going to iterate and we're going to create a tuple where we're finding the index value
for our little columns. And we need to create an if statement to
handle, if the item is in the particular column let's print out the index position.
They're right, number 40. So we're on the right track now let's do our work around fix and what do
we want to change? We want to change to do we want to change 40 let's change number two. And let's
get the name we want to change it to which would just be this. Then let's print out the column names.
Make sure it's adjusted now we're good to go. Let's check the other one position 40 is intact
and that's the workaround to change the duplicate column name versus the advertised listing you see
on stack overflow and pandas documentation which is highly annoying. But that'll be the
conclusion of this video. So please like, share and subscribe and if you subscribe turn on that
notification bell. Feel free to hit me up on my socials here. Don't forget i have affiliate links
for amazon as well as ways to help support the channel buy me a coffee, super thanks and channel
memberships. I have a Patreon but, i haven't used it in a very long time. Throw any comments or
suggestions for future videos down below i can't wait to hear from you see in the next video. Bye
hey kid don't ever let them get inside your head they'll tell you what to do in life instead
of everything you know that you can get don't let them guide your life towards regret i'll fight