Hello, everyone, my name is Jie Feng, from Tsinghua university, today, I will introduce
our paper PMF: A Privacy-preserving Human Mobility Prediction Frameworkvia Federated
Learning.
Mobility prediction problem is to predict the next location with knowing the historical
mobility trajectory. Sometimes, it also require to predict the corresponding visiting time.
Here, we focus on predicting the next location. Following the recent successful practices
from community, we find that model with better performance will also require more concentration
of the training data. Ten years ago, we build one model for one user and now we build a
centralized deep learning model for all users by training with all the data.
While performance of previous models become higher and higher, the centralized requirement
of training data becomes an risky choice. Because individual mobility trajectories are
sensitive and have many private information of individuals. Thus, we need to build a privacy-preserving
mobility prediction model. And It should achieve competitive performance of current state-of-the-art
model but also protect the user privacy effectively.
To achieve this, we first introduce the basic deep learning based mobility prediction model.
It first accepts the discretized spatial-temporal point as input and use embedding table to
translate them into vectors. Then, we concate them together and use sequential model like
LSTM to capture the sequential transition in the trajectories. Finally, we use a linear
layer to project the hidden vector into the final predicted location. Based on this basic
prediction model, we apply several techniques to protect the user privacy and improve the
prediction performance.
First, we apply federated learning framework to preserve the private user data in the local
device. As left figure shows, in the conventional workflow, each user upload their data into
the cloud center and then obtain a well-trained model from it to complete the prediction task.
After using federated learning framework, no user data is uploaded, most of the training
procedure is executed in local device, we only need to exchange the model weights with
the cloud server. This is the first step for protecting user privacy by preserving it in
the local device.
Second, while no user data is uploaded to the cloud center, we indeed upload the model
weights to the cloud center. While risk of uploading weight is smaller than uploading
the raw data, it also introduce large risk for leaking the visiting location list of
user in the mobility prediction scenario. Attacker can pretend to participate in the
whole procedure as normal user and then observe the changes of location embedding table of
target user during training. In this way, attacker can obtain the information of visiting
locations of target user. To protect user from this kind of attack, we propose to use
differential privacy to add noise in the normal data for each user. Then, the training of
the model is divided into two stage: training embedding table with noisy data while freezing
other components and training other components with the normal data while freezing the embedding
table. In this way, the changes of embedding table is not deterministic and attacker can
not easily find the right location visited by user.
Third, due to the federated learning and differential privacy operation, the performance of model
is reduced. To further improve the predict performance and make it competitive to the
state-of-the-art model, we introduce the personal adaptor in local device for personalized fine-tuning
after the normal training procedure. It works as a additional vector before the final output
projection or correlation query and it is trained and executed in the local device to
capture the personal pattern.
We use three mobility data to evaluate the performance of our model, including two check-in
data and one dense trajectory data. Here are the basic information of these data.
Here is the result, we can find that, compare with the personal model which protecting privacy,
our model achieves better performance. Compared with the joint model, our model can also achieve
competitive performance while protecting user privacy.
Here, we present the quantitative results of protecting privacy. We can find that, compared
with directly protecting data with differential privacy, the performance of our model is more
stable when increasing the privacy requirement. It means that our model achieve a better trade-off
between the performance and the privacy protection.
As a conclusion, in this paper, we investigate the privacy-preserving mobility prediction
problem and propose several effective strategies to protect the privacy while preserving the
prediction performance. In the future, we need more works on developing efficient and
effective optimization methods for the whole system to further improve the performance
while protecting the user privacy.
That's all. Thank you.