hello my name is Katerina Renko I am a
research associate at the University of
Edinburgh and today I will be presenting
a research study called exploring the
future of data-driven product design
which was published at Chi 2020 it is
part of a larger project called cherry
factories which was funded by the EPSRC
and I will be presenting the work on
behalf of my co-authors dr. daniel
Barnett
James Thorpe dr. Daniel Richards and dr.
Dave Marie rust the work was done in
collaboration between the universities
of Edinburgh and Lancaster University
was released onto the market physical
products can be used in context and for
purposes that designers may not have
anticipated and insights from these
contacts can be very valuable for design
research we see an opportunity to
utilize data driven approaches such as
those used within the digital products
fears such as a/b testing and analytics
around IOT products and connected
devices this work aims to map the design
in the space of choices available to
design teams and it was done through a
series of workshops there is a drive
with an industry to make use of data
collected from the wild and it has an
impact on maintenance however it has a
limited reported impact on design more
specifically in contrast within the
field of HCI we see data collection from
products in the wild as quite
commonplace for design research and
examples of this is contingencies
experience sampling where triggers from
data are used to generate response
requests participants or ethno mining
and thing ethnography where collected
data in the field supports contextual
inquiry and data enable design where
technology probes collect qualitative
data alongside quantitative data in
order to support design research however
there are issues with implementing these
research methods within industry which
is more generally criticized for
utilizing data collection which
perpetuates practices that are seen to
disempower consumers such as
surveillance capitalism and assumption
traditional physical product design site
extent the moment that a product is
deployed in contrast when it comes to
digital products the data collection in
research continues long past products is
deployed onto the market within physical
products data has been seen to be very
beneficial and the development of new
designs such as in the holding
enrollment we're second iteration of the
design was generated after three
thousand stage accidents and data
collected from two thousand hours of
cyclist data this is an example of
attitudinal research we conducted three
workshops with 32 participants total
from industry and academia who reported
having expertise in the IOT and product
design each workshop was three hours
long and they were conducted across the
United Kingdom in Lancaster Edinburgh
and London in order to recruit a varied
cohort of participants participants did
a series of exercises in groups of three
to five people and the exercises were
centered around large a1 sheets of paper
on which we collected data each table
had an audio recorder on it and the data
that we collected included the audio
transcriptions from each table the all
of the written notes from the different
groups and everything was transcribed
and thematically analyzed the workshop
structure included a demo talking to
objects section future vision challenge
implications and discussion within the
demo we presented work that we have
already done within the field we're
connected Bluetooth speaker sends data
to its virtual representation or also
known as a digital shadow or a digital
twin in the next exercise the talking to
objects exercise we ask the participant
to generate a series of questions that
can be answered through data collection
around the object and as you'll see
throughout the slides I have an image of
a kettle with a bunch of sticky notes on
it which was generated through this
exercise in each workshop we had a
different object to be annotated so in
Lancaster we had the kettles in London
we had audio recorders and then
we had an electric scooter after that we
had the future vision challenge where
participants spoke about objects
products that they are actually working
on or they imagined an object that they
might be designing and then they
generated a series of questions around
that that could be answered through data
and this is an example from one of our
participants in the Edinburgh discussion
that's character so a Marty is a robot
the help show people there but robotics
engineers miles and within the primary
sector school context there's many more
to come the robot so if the lightning
dogs were the eyes to be inching lights
so commonplace for the robots we broke
through the boxes there's like yeah so
it twice is to move so what pockets
doing try a little twisted into the ring
so what we have to teach them with the
server's have some sense of how things
professional officers really understand
the force officer decision currently
sake service and we can oversee
diversity applying under special so
really identify that the main reason
this was Hitler
the plastic afterwards we conducted an
implications discussion where the
participants spoke about the
implications of implementing a
data-driven design approach within their
current personal work and finally we had
a group discussion where everyone from
each individual workshop came together
and we spoke about future opportunities
for data-driven product design more
generally and from all of the
transcriptions of notes and audio
recordings we gen matically analyzed
them and generated eight themes the key
findings in the space are around light
data gathering generating design insight
product development and redesign user
centric design support and service
products and issues so the first thing
theme that we're going to talk about
today is like data gathering and here's
the little robot that our participants
spoke about so the experts felt that the
primary use of data gathering from the
wild is to capture sensor data which can
capture how users are using the objects
wear and tear functional failures and
contexts of use so here P 16 reflected
what you would want to do is identify
people within that population who you
think are otherwise similar so that you
can understand when the profile one
changes is that because of a factor that
is particular to them or is it because
it is affecting everybody then the
participants reflected on how they might
actually generate insight from this data
here they identified that a data-driven
approach would be very useful in the
exploration of specific design research
questions such as when conducting an a/b
test or a usability test or an
ethnographic study they also had ideas
about how to make sense of the data such
as third use of machine learning maybe
when you go into work you would see the
last 24 hours of activity and maybe
there's an opportunity to create
collaborative dashboards or to even
engage consumers
within feedback and actually engage them
in questions around what the data
actually means here a key quote from a
participant was from p32 who said you
have taken a recommendation you have
looked at the compromise you match that
against your brief you virtually design
it you test it you prototype it you
submit a change request you get feedback
within the theme of product development
and redesign the participants speculated
that in the future algorithms would be
able to take care of the center of this
hypothetical data bell curve where it
could be a doll curve connected to how
the majority of consumers are using a
specific device what times they're using
it at and so on and in the future
designers might instead focus on the
fringes of that data set are there needs
that are currently not being met by the
current design of a device is there are
some changes that can be made that would
cater to certain ethnographies or groups
or needs in here peek to reflected is
your job to look at the data and the
machine learned that the machine
learning is generating or is your job to
just look at the outliers within the
theme of user centered design the
participant speculated that a
data-driven design approach would
actually enable a true user centered
design where participants consumers
would be engaged in creating knowledge
and they might actually be created in
the process of Co design in co-creation
here P 11 reflected why don't we get
users to design the product that they
need and provide the mechanisms with
which they can do that
another thing that emerged was
supporting and servicing products and
here's some interesting examples came
from participants who had worked within
the aerospace industry where for example
what rolls-royce jet engines aren't
actually sold their leased companies and
they collect vast amounts of telemetry
data which is used in maintenance this
telemetry data is currently used
post-hoc
however participants reflected that a
data-driven process would actually
encourage companies to build lasting
products that can be easily maintained
in order to actually reduce the cost of
servicing top-3 reflected I hope to be
able to use the live data streams in
order to debug it is overheating has
someone smacked it these things would
help me resolve issues quicker and offer
better customer service as a result
however the participants did identify
many ethical issues with the process
some connect to disempower you
disempowering users by reducing their
privacy manipulating their behavior and
limiting informed consent by making data
collection into a black box
similarly this process might disempower
designers as well by handing over
creative practices to data scientists
and algorithms here Peter reflected if
you find out that actually people need
better tires because they're driving
over the speed limit
do you then produce tires that encourage
people to go faster and if you don't do
you then become responsible for advising
them to buy tires that don't meet their
needs within the theme of implementation
issues the participants identified that
there's significant costs around sensor
izing products data storage and analysis
furthermore there were issues around the
need for more flexible manufacturing
processes and around data quality and
bias be too reflected if we say that
we're a software company and we're going
to get this data in the next five years
they're going to say that you're
investing in the next bar the next
Google but if you say I'm going to make
kettles and they've got data in them
they're going to ask how much or your
production costs and how much are you
going to sell the kettles for finally on
the theme of utopian future we saw the
participants were able to envision a
future where everyone benefits including
businesses creatives and consumers
consumers get well designed products
that cater to their needs creatives get
a valuable insight around behaviors in
the wild and producers benefit by
creating desirable
products within this theme some
participants shared a positive stories
such as an example from Nike who are
able to gauge when there was a flu
epidemic from data that they were
gathering from their devices and in
relation to that p3 reflected I think
we're going to have a world of Sun and
gemstones in a way where every little
optimization that we get from this way
of thinking is like a grain of sand and
overall we get a beach that we all like
but occasionally we will get a few gems
like the Nike example where you have
these wonderful things that bleep us
forward in a big way
the discussion has been shortened for
the purposes of this presentation
however a key point that we would like
to make is that we feel that there's a
potential to actually expand notions of
data-driven product design from where
they currently stand which is around
optimization and maintenance we've
proposed that a desirable data-driven
design tool would enable designers to
actively investigate research questions
and this can be done in two ways firstly
by sensor izing prototypes used within
design research and secondly by
utilizing data from connected products
post manufacturer to inform the early
stages of design of new species or
versions of that product finally a few
reflections on the implications of
conducting data-driven product design in
practice in industry and academia
context this process was plausible to
participants in cases where data
gathering is already part of the
function of the device or within a
service design model specifically and in
order for this process to be acceptable
it needs to be clearly differentiated
from surveillance with negotiated proper
informed consent and with responsible
with a reasonable burden for
participants to understand the data
collection process practices should not
just be focused on quantitative sensor
data but that should also be ground
truth through qualitative data and
finally not every product should be
censor eyes and not every product should
be collecting data instead we see an
opportunity to censor eyes a subset of
devices are a subset of prototypes which
can be used in research in conclusion
the work presented here primarily
contributes a current industry and
academia perspective on the future of
data-driven product design it explores
what might happen if data from devices
in the wild is used for design research
and sense making the experts felt that
usability tests a bee test and
ethnographic studies can all be
supported through a data gathering from
products in the wild in turn this
data-driven approach can enable the
redesign of hardware and software and
enable a truly user centered design
however issues still remain around the
ethics privacy security and cost of
implementing a data-driven design
process finally I would like to thank
all of our participants for their time
the tidy factories project digital
catapult for hosting our event and
advertising it in London and for census
for advertising our event in Scotland