Hi everyone - my name is Sarah, and today I'll share our project:
Are We Healthier Together? Two Strategies for Supporting Macronutrient Assessment Skills and How the Crowd Can Help (or Not).
Unhealthy diets are linked to a number of diseases, including diabetes, cardiovascular diseases, certain cancers, and more.
A key reason why people make unhealthy food decisions is low nutritional literacy.
The ability to estimate the nutritional composition of a meal - such as the number of carbohydrates it has -
is often an important part of managing chronic health conditions.
However, learning macronutrient assessment skills is not an easy task.
The standard approach in diabetes education for teaching these skills is tedious
and involves many multi-step calculations.
Complex meals with many components add another layer of difficulty,
making the process difficult even for individuals with experience.
Prior work has explored how users might pass off the burden of assessing their own meals to an app,
such as an app that supports manual food logging or automatic ingredient identification.
However, there is currently a tradeoff between the burden of logging meals and the accuracy and completeness of meal logs.
Further, studies have shown that selecting meals from nutrition databases often leads to inaccurate records,
as untrained individuals often select meals that only loosely resemble their own.
Even if logging provided accurate results consistently for in-the-wild meals, users have reported becoming frustrated at the need
to photograph all their meals prior to eating, essentially watching their food get cold and requiring patience from meal companions.
Prior work has proposed that the wisdom or data of the crowd might be applied to lessen the burden of making healthy food
decisions, and even support the building of macronutrient assessment skills over time.
We built on this idea of using the crowd to support macronutrient assessment skills in two studies.
In our first study, we were interested in whether the crowd could provide enough information to lessen the burden
of individual macronutrient assessment.
In our second study, we developed two approaches to teaching nutrition estimation,
referred to as Decomposition and Comparison.
Both approaches used crowdsourced meal photos as a means of teaching nutritional assessment skills.
Study 1 relied on an existing dataset of 73 “in-the-wild” meal photographs.
These photos were contributed by non-expert users, and annotated by a team of 20 registered dietitian experts.
Expert annotations included macronutrient assessments for each meal,
as well as the ranked suitability for each meal photo toward achieving a particular health goal.
We then developed a mobile app to test our hypothesis that lay users might be able to make healthier meal choices for a
particular health goal if they could see what the crowd believed to be the correct choice.
Through the app, 68 users participated in five rounds in which they attempted to select a meal photo
that best fit a particular health goal.
After they made their selection, users could view what other members of the untrained crowd had selected as their choices
through a Community Board.
The Community Board also displayed why each member of the crowd made their choice.
After viewing the crowd’s choices and reasoning, users were free to change (or not change) their answer.
Through our analysis of this study’s results, we observed that
no statistically significant learning occurred in any of the groups from pre- to post-test.
The “wisdom of the untrained crowd” did not appear to be enough to support healthy food decisions.
Following Study 1, we then considered how a digital app using crowdsourced information could be
designed to better support macronutrient assessment skills.
The same dataset of annotated meal photos from Study 1 was used for Study 2.
We developed two approaches that apps could take to support nutrition estimation skills.
The first approach, Decomposition, was based on a strategy currently used in diabetes education.
In Decomposition, also known as carb counting - a meal is broken down into its macronutrient components.
This method, however, can require many steps, particularly for complex meals that include multiple ingredients.
As a result, this approach to nutritional estimation can be burdensome and
historically has led to abandonment from users.
Our approach mirrors this process but is scaffolded through the app structure and organization
to help reduce these burdens while users learn and practice these skills.
The second approach, Comparison, involves practicing comparing one’s own meal to comparable “in-the-wild”
crowdsourced meal photos that contain nutrition information.
This approach was designed to be less cognitively taxing than Decomposition, and also provide a way
for users to view inspirational meal ideas from the crowd at the same time.
We were interested if these strategies were more effective compared to a control group
(which provided no learning opportunities at all), and whether one strategy led to better learning than the other.
We recruited 93 users to take part in Study 2.
While participants in all conditions recognized the presence of a learning curve, the majority in both experimental groups
believed their approaches to learning about carb estimation
were effective in preparing them for a meal assessment post-test.
Members of the control group, on the other hand, found the post-test difficult and time-consuming.
The results of the post-test further revealed that individuals who practiced either the Comparison or Decomposition strategy,
had, indeed, resulted in learning.
As a bonus, users in the experimental groups reported being encouraged by the learning process
and inspired by the crowdsourced photos to try new meals befitting their own nutritional goals.
Interestingly, the Decomposition group was much more negative about the potential inaccuracy of their estimations.
On the other hand, the Comparison group participants reported their experience as beneficial despite the fact that they were
working with estimates instead of exact numbers.
Overall, our results from Studies 1 and 2 suggest the following key lessons learned for the design of future apps.
Lesson 1: The crowd doesn’t necessarily teach macronutrient assessment.
Users did not appear to reliably learn macronutrient assessment skills
through viewing community decisions and justifications.
In fact, an untrained crowd might harm learning in certain contexts.
Nutrition education apps with a crowd component should take care to evaluate accuracy and trust in their situational contexts.
Lesson 2: When considering how to design digital apps for nutrition education, carb estimation skills appear to be gained
through either the Decomposition or Comparison approaches.
We did find that the Comparison approach appeared to result in more instances of learning across food categories,
as well as more positive reports of the experience by our users.
The Comparison strategy thus appears to be at least on par with the traditional Decomposition strategy,
and may even be more beneficial for users in some respects.
Finally, both studies reveal insights into how we might design an app to develop trained experts in nutrition.
Our third lesson was that crowdsourcing meal photos and labels from app-trained experts in this way may be a powerful
approach to support future macronutrient assessment education and inspire healthier eating.
Future apps which are able to successfully leverage the findings from lesson 3 may be able to encourage users to more
regularly practice their macronutrient assessment skills, discover new healthy options that work best for them,
and enjoy the process overall.
Thanks for listening to our talk today, and special thanks to the nutrition experts, study participants, reviewers,
and all others who supported this work.