Hi! My name is Maximiliane Windl, and I am very excited to present our paper
Automating Contextual Privacy Policies: Design and Evaluation of a production tool
for digital consumer privacy awareness.
The tool is called PrivacyInjector and helps remedy the current limitations of privacy policies
by displaying shorter snippets of the privacy policy directly in the user's current context.
In real life, no one would just sign documents or contracts offered by anonymous persons without even reading them.
But that is exactly what happens every day on the internet.
To this day, Privacy policies remain the most established way to inform users about their rights and choices online.
However, most users do not read them but agree to the terms nevertheless.
Reasons include their length, difficult legal language, abstract wording, and inconsistent placement.
Research proposed contextual privacy policies to overcome these issues.
They embed the relevant text snippets directly in the user's current context,
making the required information available - just in time and just in place.
However, contextual privacy policies are currently only implemented as a concept showcase for 7 different websites.
At the moment, the creation and display of contextual privacy policies require a lot of manual effort,
which hinders their adoption in practice.
Those manual steps include the identification of areas on the websites that are suitable
to display privacy policy information, the automatic detection of the privacy policy,
the extraction of meaningful snippets, and last, the display of the information.
Therefore, this work evolved the contextual privacy policy into PrivacyInjector. PrivacyInjector is a tool that
automatically identifies privacy policies, segments and annotates them using machine learning,
automatically identifies suitable contexts, and finally displays them in a way that aligns
with the website's layout.
For the Identification of the privacy policies, we first construct a list of all links on the website
that have a privacy-related keyword in their URL. We then apply a set of RegExes to these websites
to search for phrases typically contained in privacy policies.
For the segmentation and annotation, we first try to break the text up according to its div and p tags.
Whenever that first step fails to return adequate segments, we employ an algorithm that uses word embeddings
to extract semantically coherent sentences.
We then use hierarchical multi-label classifiers to annotate these segments with their affiliated data practices.
Multi-label means that one sentence can have multiple data practices. For example, the sentence shown here:
"We will collect your phone number and payment information." gets annotated with three data practices in total.
Those are: 1st party collection, collection mode - explicit, and information type - contact and finance.
We developed various techniques to identify the contexts to display the CPP-elements.
Those included searching for HTML-Ids, CSS-Classes, plain texts, or keywords in URLs.
For this work, we focused on six different categories of data practices. Those were:
Advertisement information, information about users with accounts, financial information, cookie information,
information about data collection and sharing with social media platforms, and location information.
We chose these categories because they are, on the one hand widespread practices that have
a high likelihood of being covered in the privacy policy. On the other hand, these categories are also
often represented as elements on the website.
We used an iterative design approach for the design of PrivacyInjector. Because PrivacyInjector is supposed to
automatically fit with all kinds of various layouts, we use floating, draggable bubbles in combination with collapsable
sidebars to display the texts.
We conducted two empirical studies to evaluate PrivacyInjector. We first evaluated the
technical functionality on 500 of the most visited websites in the United States with 59 participants.
We then did post-study interviews and improved the extension based on this feedback.
Afterward, we deployed the extension for 10 days with 11 participants. The participants were instructed to keep on
using their computers as usual, but with the extension installed. We did pre-and post-study interviews,
and the participants filled out questionnaires. We had to exclude several of these most visited websites
due to issues including them not being in English, showing violent or explicit content or the privacy policy
being unconventionally presented, for example, in a pop-up, which made the automatic detection impossible.
This reduced the initial pool of websites to 354 websites in total.
For the first study, the participants had to rate for each element whether the placement fitted with the category,
whether the category and the texts matched, and whether the element and the sidebar were fully visible.
This study revealed very satisfactory results as can be seen in this chart. However, the placement of
location bubbles sticks out. This led us to conduct a more elaborate error analysis.
For this, we investigated every bubble that was rated as misplaced by our participants.
Thereby we found three reasons for misplaced bubbles: The first one is semantic mismatches.
This happened, for example, when the word cookie was recognized on recipe websites where it
nothing to do with the technical cookie. The second is reloading elements. That was a problem for advertising
elements that reloaded multiple times while the user was on the website. Whenever such an element disappeared,
the bubble we had placed on the advertisement element disappeared with it and the last reason was wrong judgments.
Sometimes, it was not immediately apparent that a bubble was correctly placed. For example, amazon has a link
in the footer that links to the login for merchant accounts. We assume that participants did not view this as a login
element and therefore rated the bubbles as misplaced.
After we improved the extension based on the findings from the error analysis and the post-study interviews,
we conducted the second, in the wild study. We analyzed the qualitative interview data using thematic analysis
and found three main themes: Awareness, presentation and interaction.
The theme awareness describes users' perception of privacy relevance. Here, participants reported that the tool
surfaced hidden ads that were disguised as information and one participant even reported leaving a website
that wanted to share location data with third parties. Overall, participants agreed that the extension
increased awareness and helped to make informed decisions.
Presentation covers a wide range of suggestions and comments regarding the presentation of policy snippets,
including wishes for even more concise information, interpretations of the severity of privacy threats,
or highlighting updates. Interaction describes experiences around regular tool usage. Here, participants mentioned
several areas where they found PrivacyInjector most useful. Those were, for example at registration to learn more about
account practices, when information was perceived as surprising, or on frequently used sites as users expected
to leave most data there.
Overall, we can report that PrivacyInjector empowers users to make informed decisions.
and Privacyinjector helps translate existing privacy concerns into concrete behavior change.
However, future work should try to further reduce the text length and visualize the severity of privacy threats.
And since our tool showed that several websites have barriers that hinder the automatic analyzation
and communication of contextual privacy policies, we encourage service providers to foster
machine accessibility of their privacy policies.
We hope our work inspires further research on contextual privacy information
and if you have any questions, please get in touch. Thank you so much for your attention.