Understanding Privacy-Related Questions on Stack Overflow
Hi my name is Mohammad Tahaei. I'm a PhD candidate at the University of Edinburgh.
I'll be talking about our paper Understanding Privacy-Related Questions
on Stack Overflow. This project has been done in collaboration with my
supervisor Kami Vaniea and my fellow PhD candidate Naomi Saphra.
So why privacy? Why Stack Overflow and why this research. The story starts with software developers
who are the main work force of our digital society. While designing and coding
software they make decisions that can impact not just the software but also
the people who use the software and of course some of those decisions will
impact security and privacy of the users. Here we are interested in the privacy
aspects of the software. While it would be wonderful if all software respected
users privacy that's not always the case. For example right now there are
discussions about the use of an app which can trace people for COVID-19
reasons. So as long as we use software in our daily routines privacy is an issue
that we need to talk about. Since not all developers have access to security and
privacy experts and even some of them rely on privacy unfriendly methods such
as ad networks we're keen to explore the intersection of privacy and software
developers. Even though there is research about security practices of software
developers but there is little known about their privacy practices.
I'd like to start by defining privacy just to get us all on the same page. What is
privacy? As Daniel Solove puts it privacy is a complicated concept to be boiled
down into one sentence. Hence he proposes for activities that are harmful to
privacy. These activities are information collection, information processing,
information dissemination, and invasion. When privacy enters
engineering world it's defined as a set of requirements.
When it comes to software engineering there are several frameworks to make
privacy tangible in the software development process such as privacy by design
which talks about concepts like minimization of data collection or
informing the users about data processing activities. However privacy by design
is also not fully adopted by developers. Developers see privacy as a
security related issue or as someone else's job one. While prior work uses
interviews to look at the use of privacy by design for developers we decided to
expand the literature by doing a bottom-up, artefact analysis.
We studied Stack Overflow. Because Stack Overflow is one of the largest Q&A platforms for
software developers. Developers ask questions and learn from Stack Overflow
or even copy code from this platform and paste it directly their projects. Several
papers show the value and importance of this platform in other software
development areas such as security and data management therefore we decided to
focus on this platform.
Our research questions pinpoint two areas one what
topics do Stack Overflow users associate with the word privacy? Here we look at
how Stack Overflow users define privacy, instead of a top-down approach and
trying to fit Stack Overflow questions into a framework. And the second question
is what or who is pushing Stack Overflow users to engage with privacy related
topics. The idea here is to look at what causes them to ask a privacy related
question on Stack Overflow. For example while they're programming because
of a compiler error they turn to Stack Overflow to understand and find a
solution for that term. We were keen to explore such causes
for privacy related questions. To answer our research questions we use two
methods. First we use topic modelling which is an automated way of finding
topics in text. And second use a qualitative approach to dig more into
the data. In this presentation I only focus on the qualitative part because
of timing issues. Please refer to the paper for the topic modeling section and
to find out more about how these two methods compare with one another.
Alright, how did we collect the data.
We collected all questions that had the word primacy either in their tags or in their titles.
In the graph on the right SO-all represents all questions on a Stack Overflow with over 18 million
questions. SO-privacy shows all privacy related questions with 1733
questions and SO-privacy-rand is a random subset of all privacy related
questions with 315 questions. We use SO-privacy in our topic modelling section
and SO-privacy-rand in our qualitative section.
Here is an overview of all privacy related question. In the graph on the left side you can see the
increase in the number of privacy questions over the past decade and on
the right side we can see which words are commonly used as tags by Stack
Overflow users. Facebook has been used a lot not to our surprise, mobile
operating systems such as Android and iOS are also visible.
Now let's dive deeper into the data with qualitative analysis.
For the rest of the talk I will focus on the 315 questions we randomly sampled from all privacy related
questions. We further excluded 21 questions because they were about public
and private variables or were too vague to understand. The final includes 294 questions.
In the qualitative analysis will look
at four elements in each question question type which is based on the
prior work and existing taxonomies such as how do I fix this
or what does this error mean. Driver which looks at what pushes the users to
ask a question for example a compiler error or a client requirement for a
Facebook warning. Accepted answers and privacy aspect of the question. Just a
quick note that some questions might have various angles but we only focus on the privacy angles here.
For the question types we rely on the prior work on Stack Overflow.
As you can see many questions not to our surprise are around how. For
example this user is trying to remove their personal email from Git logs and
is asking how to do it. Interestingly many people are asking questions that
are abstract or conceptual. This may signal that users are asking questions
about privacy related topics in the earlier stages of software development
and not necessarily in the implementation phases. For example this
user is trying to understand the privacy related consequences of using CDN
services. For the other two, error type represents an explicit error in the
question and unexpected category represents questions when users expected
some behavior from the system but encountered something else.
Drivers are one of the fascinating results of our work. They showed on what
or who makes users to turn to Stack Overflow for asking questions. Users come
to Stack Overflow to ask questions because of personal concerns. For example
thinking about how to minimize data collection. Here you can see a user who
is looking for suggestions for designing a database with multiple access levels.
The other even more interesting driver is platforms such
Facebook, Google, or iOS. Even though in terms of number they stand second but it
shows their power and authority over privacy. If they define a resource as
sensitive or introduce a new privacy related requirement developers have to
follow those. Because they want their app published.
For example this user has encountered a privacy policy issue while submitting an
app to app store and is trying to find a solution for the problem. The third
player here is laws and regulations such as GDPR which didn't appear strongly in
our data set, potentially because of the technical nature of Stack Overflow.
130 of a random sample questions have an accepted answer meaning that the asker
marked them as accepted. Interestingly a lot of questions are being answered by
Stack Overflow users without any links to other resources which means the
community is building knowledge around privacy related topics on Stack Overflow.
This is similar to what prior research shows about Android documentation.
Stack Overflow covers a major part of Android documentation in a Q&A format which is
easier for developers to digest.
We thematically analysed the questions for
their privacy aspects. You can see the eight emerging themes in this table. For
example access control includes 119 questions with over 100,000 views some
questions in this theme are how to set privacy permissions for various social
networks or how to design a database about multiple access levels. For example
this user is asking about how to set a privacy related field in Vimeo's API.
Privacy policies cover topics about how to write and add privacy policies. For
example this user is trying to publish their app
on Facebook but is not able to because a privacy policy is required from Facebook.
Developers with privacy concerns represent a group who is concerned with
privacy or is trying to find a way to follow privacy by design practices. For
example in this question the user is asking about how they can trust a
library provided by Facebook. On the other hand we observe developers who are
going the opposite way and are asking questions that are privacy unfriendly
such as how can I collect more data about the user.
So what are some implications of our work?
First platforms such as Apple and Google are some of the main
drivers of privacy they decide which apps can be published and which cannot
they can add or remove to what is considered as private and sensitive any
changes in these platforms could have a huge impact on privacy. You can see both
Apple and Google require developers to follow their privacy requirements.
privacy policies are challenging for developers. Some of the potential causes
of this could be not knowing why privacy policies are necessary or not knowing
what language they should use or even if they do know the process of adding one
is complicated and makes the whole procedure difficult.
And here is the famous saying in the community "developers need support" In particular
for privacy they need tools that can identify third-party libraries with
privacy leaks. Tools that can walk them through privacy policy creation tools
that can automatically detect and adapt to updates
and better integrate in their natural workflows such as integration in IDEs.
In summary our work explores challenges and confusions that
Stack Overflow users have around privacy related topics. We found that platforms
such as Apple and Google are some of the main drivers of privacy. We further
highlighted several privacy related areas that developers need support such
as privacy policy creation tasks
and integration of privacy checks into code development processes.
Thank you very much for listening.