Software developers, we are not known for our rocking style, are we?
Or are we?
Today I'll show you how I use ML to make me trendier,
taking inspiration from influencers.
I'm going to build an app that analyzes my closet,
and pictures of fashionable influencers,
and then makes me outfit recommendations.
If I'm missing an important piece,
it'll also recommend me products to buy.
Let me show you how.
♪ [music] ♪
So the way this is going to work
is I'm going to grab a bunch of fashionable inspiration pictures
from social media,
and I'll take a bunch of pictures of my wardrobe,
and I'll try to match the pieces that I already own
to those fashion pictures using Machine Learning,
which will help me put together outfits.
I'll wrap this whole thing in a Firebase app
so I can scroll through outfit recommendations on my phone.
All right. Let's build this thing.
First, I needed to catalog my entire wardrobe.
I thought I could just grab a few pictures of me
wearing clothing in everyday life
and then use computer vision to pick up the items.
To do this I used a feature of the Google Cloud Vision API
called Object Detection, which identifies objects in photos
and then draws a little bounding box around them.
As you can see, it kind of worked.
It identified my top, shorts, shoes, stuff like that.
But since I'm going to be matching my clothing pictures
to inspiration pictures,
I realized that the quality of my pictures had to be really high.
They couldn't be at weird lighting angles or anything like that.
So I accepted the sad fate
that I was going to have to take a picture of every single item in my closet,
which I did painstakingly over a couple of days,
dressing and undressing my mannequin
and taking pictures of all my different shoes.
Next, I needed to find some people
who actually knew something about looking trendy,
and bonus points if they also worked in tech.
So I reached out to Laura Medalia.
Laura is a developer who posts
lots of fashionable outfit picks of herself on Instagram,
where she is @codergirl_, and from head to toe,
which is important because we need to capture those shoes.
With her permission, I downloaded a bunch of her pictures
and uploaded them to Cloud Storage.
Again, the goal here is to take a bunch of Laura's pictures
and then pictures from my own closet,
and see which of her outfits I can kind of recreate.
Now, challenge one was that not all Laura's pictures
are actually of clothing,
so, first, I had to eliminate those and select only the ones
where she is in an outfit.
For that, I used a feature of the Vision API
that does image classification.
This feature returns a bunch of tags describing what in general
is going on in an image,
and one of the tags it returns is
I sorted through all of Laura's pictures and only kept ones
that had a person in them, and they were also labeled
And with that, I had my inspiration image dataset.
Now, the next step is to implement the feature that looks at my closet
and the inspiration picture and matches them.
For that, I'm going to use yet another feature
of Google Vision API called Product Search.
If you're ever scrolling through a store,
and you're looking at a product, and you see similar items,
that's the functionality
that the Google Vision Product Search feature gives you.
The next step is to create a product set.
For this, you want to use the
The first step is to create a
that connects to the Google Cloud backend,
and next, you'll create a
where you can hold all of your clothing items.
I called mine "dales_closet."
Next, you want to go to the spot
where all of your clothing items are stored.
As you can see, I've got all of mine in a directory,
and if I want to add something like this black bomber jacket,
in the code I create a new
I name it, I give it the category
and then I add any labels that I might want to later go back and filter by,
so just show me jackets.
And the next step is to teach the model
what this bomber jacket looks like.
For that, you want to add paths to all of the different files
as reference images.
It's better if you can take pictures from different angles
with different lighting conditions,
because it will help the model generalize better.
To make this a little bit easier for you,
I wrote a script in Python so that if you organize your images
into a directory structure,
then you can just run this script,
and it will automatically create a product set for you.
Make sure you check out the blog post to see how.
Once you're done creating your product set,
it'll take about 30 minutes for the API to actually index those images.
So just hang tight.
When it's finished, we get to actually make outfits with ML.
To do that, we'll call the
and pass it a path to an image file.
First, it analyzes that image and identifies any clothing it finds,
then it looks through your closet and finds the best matches.
And now, here's where it gets interesting.
Because even though the API returns back similar items,
what we really want are outfit recommendations.
An outfit like has only one pair of shoes,
and you can't be wearing a skirt and pants at the same time, probably.
So we need to do a little bit of finagling.
And that's kind of par for the course with ML models.
They usually don't give you back exactly what you want,
so you have to use your expertise and domain knowledge
to make sure the predictions make sense.
So I'm going to roll my own outfit algorithm
that defines rules for what pieces of clothing can be in an outfit.
And it'll take all of the confidence scores
returned by the API, and generate one new quality score
for the entire outfit.
I did all of my outfit-making logic in a Python script,
which wrote out all of my outfit recommendations to Firestore.
Then I built a little frontend app
that pulls in all the recommendations, sorted in score order.
Let's take a look at how I did.
♪ [music] ♪
If you want all the details on how I built this app,
make sure you check out the blog post link below.
And follow codergirl_ on Instagram
to get your own neat techy outfit inspo.
See you next time.
♪ [music] ♪