PRIYANKA VERGADIA: We've all experienced
the benefits of crowdsourcing.
Whether it's restaurant reviews that
help us find a perfect place for dinner or crowdfunding
to bring our favorite TV show back to life,
these distributed contributions combined
to make some super useful tools.
What if we use that same concept to build
better machine learning models?
Stay tuned to find out.
[MUSIC PLAYING]
Welcome to AI Adventures, where we explore the art, science,
and tools of machine learning.
I'm Priyanka Vergadia, and on this episode,
we are going to examine a different approach to machine
learning.
Standard machine learning approaches
require centralizing the training data
into a common store.
So let's say we want to train a keyboard prediction model based
on user interactions.
Traditionally, we implemented intelligence
by collecting all the data on the server, create a model,
and then serve it.
Clients all talk to the server to make prediction.
The model and the data is all in one central location,
making it extremely easy.
But the downside of this centralized setup
is that this back-and-forth communication
can hurt the user experience due to network
latency, connectivity, battery lives, and all sorts
of unpredictable issues.
One way to solve this is to have each client independently train
its own model using its own data right on the device.
No communication necessary.
Great idea, right?
Well, not quite because each individual device
does not have enough data to render a good model.
You could pre-train the model on the server and then deploy it.
But the problem with that is, in our smart keyboard example,
let's say if everyone started using a new trendy word today,
then the model trained on yesterday's data
won't be as useful.
So what should we do to utilize the goodness
of decentralized data while maintaining users' privacy?
Well, that's where federated learning comes in.
The core idea behind federated learning
is decentralized learning, where the user data is never
sent to a central server.
So how does it work?
Well, you start with a model on the server,
distribute it to the clients.
But you can't just deploy to every client
because you don't want to break the user experience.
You will identify these clients based
on which ones are available, plugged in, and not in use.
Then, also find out which ones are
suitable because not all clients will have the sufficient data.
Once you've identified suitable devices,
we can then deploy the model to them.
Now, each client trains the model locally using
its own local data and produces a new model,
which is sent to the server.
The thing to know here is that the data used
to train the model on the individual device
never leaves the device.
Only the weights, biases, and other parameters
learned by the model leave the device.
The server then gets all the locally
trained models and averages them out,
effectively creating a new master model.
But how do we know that this process
is doing something meaningful and actually creating
a good model?
Doing this once is not enough.
We have to do it over and over so the combined model becomes
the initial model for the next round.
And with every round, the combined model
gets a little bit better thanks on the data
from all those clients.
And many, many rounds later, your smart keyboard
begins to show signs of intelligence.
Now, we just saw that all the training data remains
on your device and no individual updates
are stored in the cloud.
For additional privacy in federated learning,
we can use the concept of secure aggregation, where
the server pairs up devices with others in a buddy system.
For example, here, each device has two buddies.
Data from each device can be combined with the random values
before being sent to the server.
The server obviously knows the value sent to the buddies
and cancels them out to just get the payload.
This trick obfuscates the data while in transit to the server.
If you've used Google Keyboard, the Gboard, then you
have already seen and experienced
a federated learning use case.
The Gboard shows a suggested query,
your phone locally stores information
about the current context and whether you
clicked the suggestion.
Federated learning processes that history on device
to suggest improvements to the next iteration of Gboard's
query suggestion model.
In this episode, we learned that federated learning
is a collaborative and decentralized approach
to machine learning that allows for smarter models, lower
latency, and less power consumption,
all while ensuring user privacy.
Federated learning is still a relatively new concept,
and there's definitely more to come in the future.
I would love to know some of the federated learning use cases
that you're thinking about.
Drop them in the comments below.
And I look forward to seeing you on the next episode of AI
Adventures.
In the meantime, if you've enjoyed this video,
click that Like button and subscribe to get all the latest
episodes right when they come.