Hi, I'm Krishika Haresh Khemani and will be talking about the research
I did in collaboration with Dr. Stuart Reeves on practitioners attitudes towards codifications
of voice user interface design knowledge.
Recently, there has been an increase in HCI interest and effort towards codifications
of voice user interface design knowledge in both industry and academia.
So what do I mean by codifications of design knowledge? I simply mean documenting design
principles, guidelines, heuristics, and best practices for voice use interfaces.
Practitioners in the industry have published books documenting such design guidance,
for example, Cathy Pearl.
There is also platform-specific design guidance available.
Here is an example from the Alexa design guide. Create a simple set of options.
Avoid tasks involving information that the customer is not readily familiar with
or complex data such as long alphanumeric passwords.
They also provide examples of how to implement such design guidelines.
As I mentioned earlier, efforts by academic researchers have also been made
towards codifications of voice interface design knowledge.
Wei and Landay proposed 17 heuristics for speech user interfaces.
Here is an example of one of them Keep feedback and prompts short.
Clear but succinct. Keep lists of items short
3-5 maximum, and let people ask if they want to hear more.
Let experienced users have faster and shorter prompts.
If you noticed this example is very similar to
the previous example I showed from the Alexa design guide.
But while these efforts are being made to codify voice interface design knowledge,
it is important to ask how do voice user interface practitioners actually
conceptualise design guidelines? And how do they implement it?
This is very important question to ask as a users of such design guidance are often practitioners.
So, through our research, we attempted to answer this question.
We first conducted a preliminary survey to recruit participants for our interview study and gather
some information relevant to our research topic. Initially, we had 63 respondents to our survey,
but it was more important for us to seek participants with professional experience,
that is with real world familiarity with designing voice user interfaces for actual deployment.
42 respondents met this criterion. 16 out of those 42 respondents expressed their
interest to interview with us. But only 9 respondents actually interviewed.
As depicted here in this pie chart, all our interviewees had at least 1 year of experience.
And two of our interviewees had over 10 years of experience. These two interviewees
had experienced with interactive voice response systems, or also known as IVRs.
We conducted a semi-structured interview study, where we asked our interviewees about
their experience with voice user interfaces. What kind of design process do they follow?
And how do they apply platform-specific design guidelines, such as those by Amazon or Google?
We also wanted to know what their thoughts were on existing platform-specific design guidance.
And how did they feel about harmonisation of it?
By harmonisation, we mean bringing together various codifications of design guidance.
To analyse the interview data,
we used an inductive and deductive approach to thematic analysis.
Three major themes emerged from our analysis. First, trouble with terminology.
Second, voice user interface design process, design guidelines and considerations.
Third harmonisation of voice user interface design knowledge.
To begin with trouble with terminology. So we noticed that some of our interviewees
use the terms design guidelines, best practices, and heuristics interchangeably.
Whereas others made a distinction between them. P1 described design guidelines as a checklist
of do's and don'ts, and heuristics as a set of values. On the other hand,
P8 said things that are labeled as your best practices are don't give
anybody more than three choices at a time. Guidelines are make it clear
by structuring information in a logical order and not presenting it too quickly.
These quotes mainly highlight the distinction our interviewees were trying to make
of guidelines as prescriptions versus guidelines as values.
Moving on to the second theme, voice interface design process, guidelines, and concentrations.
We observed that our interviewees struggled to articulate how they implement design guidelines,
but they were able to speak about the design process in great detail.
P7 here says, so we do the sample scripts. If I have time, I annotate them with kind
of references to design guidelines. Once we approved the scripts or started,
usually I'm also recording myself, kind of talking through the dialogue. Once we have a point where
we feel good, the client feels good. They've heard a few recordings. They
have a good sense for how, at least he happy about it is going to sound.
That's when I go in and start to really do bulky flow chart. And I start to say,
okay, these prompts that we are providing give rise to these conversational pathways.
We noticed that our interviewees often describe their design process in ways similar to the
graphical user interface design process of discover, define ideate, design, and evaluate.
We also noted that their use of design guidelines were tied to their attitudes towards it.
For example, P1 said in those early days, I read a lot of those design guides and kind
of treated them as a set of rules. Now, I would say that I have a much
deeper understanding of design and how it's like a user experience process.
I actually don't like the concept of design guides any more because I find them very
limiting. It's about weighing those pros and cons, forming hypothesis and testing it.
Our interviewees pointed out some other factors to consider during the design process.
For example, is it a voice only or multimodal experience? Components of language, and so on.
Finally, the third theme - harmonisation of voice interface design knowledge.
Our interviewees had mixed perceptions towards harmonisation of design knowledge.
They acknowledged that existing codifications are broadly similar and had overlapping concepts,
But they also mentioned there were several other considerations to be made.
Such as platform-specific differences, language being dynamic
where for example, P8 says language continues to be very emergent and flexible.
Design immaturity. Many interviewees spoke about voice design still being a very young field.
Multimodal design guidelines being different across different platforms.
Finally, going back to the question we asked early on - how do voice user
interface practitioners actually conceptualise design guidelines? And how do they implement it?
Our research indicates that the way practitioners employ design guidelines depends on their
design competence, novice practitioners treat them as foundational knowledge.
Whereas more experienced practitioners may rely on their own experience to determine which guidelines
are fitting for a particular use case, or may even come up with their own design guidelines.
For voice user interface-specific guidelines,
or heuristics to be effective they must be adoptable by designers - Murad et. al.
As we mentioned earlier, our interviewee struggled to articulate
exactly how they implement design guidelines while designing for voice user interfaces.
Therefore we suggest further investigating how codified design knowledge is enacted in
practice through through methods, such as ethnographic research.
Thank you very much. And I look forward to answering your questions during the Q&A session.