Hello everyone, Thanks for coming to my presentation today. My name is Yuan Tian.
I’m going to talk about our work, which is Cohort Modeling Based App Category Usage Prediction.
Ok. As we know that, nowadays, smartphones have become increasingly important in our daily life.
We use it to communicate with friends, play games, take pictures and even for ordering food.
And actually, there have been many previous work conducted for predicting users’ app usage behavior.
Normally, the usage prediction problem is,
we know a user’s historical usage behaviour, like the historical usage logs,
then we use them to predict which app or app category the user will use in the future time.
As we mentioned before, the previous works actually have proposed many app usage prediction models.
However, the cold-start problem hasn’t been solved or addressed properly within those models.
Because, when a new user came, we do not have a large amount of users’ historical usage information.
In the beginning, we may only have a few recent app usage logs of this new user,
but we still need to predict which app or app category the user will use at the current time.
Therefore, in our work, we propose to leverage user-cohort information in the app category usage prediction,
especially for alleviating the cold-start problem.
So we firstly introduce the definition of the cohort, which is a group of people
who share common characteristics or experiences within a defined time-span.
Then, for an instance, the users can be grouped into the cohort based on their age and gender.
So if we got a new user, who is a male and aged 30 years old.
Even we do not have a lot of information of this new user,
we can still leverage the app usage behaviour
of the other members in the same cohort to enhance the prediction.
For example, the other users will use facebook around 5 pm,
so we can assume that the new user should also have higher probability to use facebook around this time.
Ok. Given this hypothesis, we aim to answer two research questions in our work.
The first one is: Could app usage behaviour be predicted based on the user cohort information?
If yes, which cohort information will be more important?
Then the second question is: Would user cohort-based models really alleviate the cold-start problem?
Then we firstly formulate the final goal of our work,
which is the app category usage prediction.
For each user, given the current time T and cohort information C,
we want to predict which app category this user will use.
We then step to the major part of our work, what cohorts can be used and explored in our work?
The tree graph here shows the overview of all the cohorts we explored in our work.
By following one previous work, we know that
there are three different dimensions have been used in segmenting smartphone users,
which are: demographics, psychographics, and behavioural patterns,
where demographics are based on static user characteristics,
while the latter two are based on users’ limited historical app usage information.
And then we will introduce them one by one.
The first dimension is demographics,
which is actually the most popular method for segmenting users.
In our case, it means five groups of age cohort,
2 groups of gender cohort and 3 groups of operating system cohort.
The second dimension is psychographics. In this dimension, we measure users’ characteristics
from three different aspects, which are users’ interests, way of living, and the community belonged.
We then will introduce the detailed information
for how we classify users into these cohorts.
To group users based on their different interests, we represent each user by their absolutely
and relatively top 3 app categories.
For the absolutely top 3 app categories, as shown in this example,
if we have a user’s usage logs as this, we can find that
the top 3 app categories with highest usage frequency are social, communication and music.
So this user will be represented by these three different app categories.
But we find that a small number of apps are always used more frequently, like social and communication,
so we further extract the relatively top 3 app categories based on the normalized usage frequency,
which normalizing by the usage frequency across all the users.
The second aspect is to group users based on their way of living.
We mainly characterise users' way of living based on their get-up/sleep time,
and nocturnal phone usage behaviour.
These features are all extracted from users’ logs by following one previous work.
For the get-up/ sleep time, an idle time of over 4 hours at night will be identified as the sleeping time.
Then the end of the sleeping time will be regarded as the get-up time,
the start will be regarded as the sleep time.
For the nocturnal phone usage, it will be calculated based on the total duration of
all the active periods of app usage during night time.
After we got these features, we will group users into three different cohorts based on
the distribution of each feature.
For example, if the user always get-up earlier
than the mean minus standard deviation,
he will be classified into the early-bird group.
Then the last aspect in psychographics dimension is to group users into different communities.
For getting users into different communities, we follow one previous work,
where they treat each user as a vector consists of app category usage frequency during specific time periods,
the time periods across a day and the status of weekend and workdays.
Then we cluster users based on these vectors, and finally get the communities
like night communicators and evening learners.
For the last dimension to get user cohorts, the behavioural patterns.
we mainly characterise users’ behavioural patterns from two ways,
which are engagement and revisitation patterns.
For the engagement, we will group users into five groups
based on the quantiles of the total usage frequency and total usage duration respectively.
For example, the users use apps less than the duration of 20% quantile will
be classified into the tourists group.
For the revisitation patterns, each user will first be represented by a 15-dimensional vector,
each dimension corresponds to the revisitation interval bins of different time length.
Then the clustering technique is applied to get the users into three different cohorts:
checkers, waiters, and responsives.
The checkers will always revisit the apps in a short time,
and the waiters exhibit longer revisitation patterns.
The responsives exhibit sometimes brief and sometimes long revisit patterns.
Ok. So we now have introduced all the user cohorts used in our work .
But besides solely based on one of the cohort we have introduced before,
we also want to combine multiple types of cohort together when representing users in the prediction.
For example here, a male user who aged between 25-34, and he is using the android phone,
If we want to model this user based on all these three cohort information,
we can simply concatenate the representative vectors of each cohort together.
By following this method,
any cohorts information can be combined when modeling users in the prediction.
Then the last thing we want to talk about before we proceed to the prediction
is the cohort assignment approach.
When a new user came, if we want to predict which app category the user will use,
we firstly need to get the cohort of this user.
So we introduce three different approaches here for finding the most proper cohort.
We can look at the figure on the right side, where the green circle can be regarded as the new user.
By using the first approach Nearest Centroid Classifier, we can find that
the nearest user of the green circle is user A who belonged to the red cohort.
So we assign the red cohort to this new user.
Then by using the k-nearest neighbour classifier,
where we set the K equals to 5, we can find that, these are the five nearest neighbours of green user.
And the majority of them are from the blue cohort. So we assign the blue cohort to the new user.
And the last approach is to train an additional classifier based on users’ representative vectors.
Ok. We then start to talk about our experiment results.
Firstly, we want to clarify the experiment settings in our work.
We apply 5-fold cross-validation to evaluate each model.
And we split the training and test sets based on users instead of the logs.
There are 3 different classifiers are used, where the XGBoost performs best
in the app category usage prediction.
So we only report the f1 score of this classifier later.
For the baseline, we use the model only based on temporal context features as our benchmark,
which takes all the users as they are ”the same” .
For the prediction results, here is the f1 score of the cohort performs best in each dimension
when predicting which app category the user will use.
And we first find that all the cohort based models achieve better performance than the baseline.
Additionally, we find that the psychographic
cohort performs much better than the other two cohort dimensions.
And among all the different cohorts, the interests cohort of absolutely top 1 app category performs
best, and then is the community cohort.
For the three different cohort assignment approaches we introduced before,
most of the time, the Nearest Centroid Classifier outperforms
the other two approaches.
And for the combinatory cohorts, when we observe the combinatory cohorts within
each dimension, all of the features in demographics performs best than using any one or two of
them, the best performance is Age plus Gender plus Operating System. And it’s same for
the way of living dimension, the best performance is Sleep plus Get-up time plus Nocturnal usage.
Then when we observe the combinatory cohorts across different dimensions, we find that
combine the demographics with behavioural patterns, we can also observe an increase
of 10% performance improvement.
However, for the interests cohorts, no combinatory cohorts could beat only using the absolutely
top 1 interests. It's still the best performance across all single cohorts and combinatory cohorts.
Finally, we want to demonstrate that if our
proposed cohort-based model could really alleviate the user's cold start problem.
so we randomly select 20% of users in our dataset to simulate the new users by increasing
the number of interaction logs available for prediction from 5 to all of the logs.
The Figure on the left shows the performance of our proposed cohort-based model and other
four different baselines from previous works.
We can find that our cohort-based model outperforms
both the personalized and population based baselines in any time of the cold-start.
And even only with 5 or 10 logs available, we can almost achieve the best accuracy with
all of the logs. Additionally, our model could increase steadily as more and more logs are available .
Ok. Let’s have a quick recap of our work.
In this work, we have answered the two research questions. Firstly, we find that the cohort
information can be used for app category usage prediction and psychographics works best, especially
the interests and community cohorts. We also demonstrated that by leveraging cohorts signal
in app category prediction, we can really alleviate the new users’ cold-start problem.
For the future works, we plan to do more long-term characterization of user cohorts.
And also conduct further investigation on next app prediction instead of the category-level prediction.
Additionally, we will refine our work by exploring more relevant fine-grained cohorts or profile
information when they are available.
Ok. That’s all for our work. Thanks for listening.
Please feel free to ask any questions if you want. Thank you so much.