Hello, my name is Shengchen Zhang.
I will present the research
"Patterns for Representing Knowledge Graphs to Communicate
Situational Knowledge of Service Robots."
This work is done in collaboration with my coauthors
Zixuan Wang, Chaoran Chen, Yi Dai, Lyumanshan Ye, and Xiaohua Sun.
Service robots need to not only utilize,
but also communicate their situational knowledge
of the working environment.
Knowledge graphs, or KGs,
are commonly used to represent such knowledge in robots.
Recent research focused on designing KG interfaces for expert users.
This helps experts to understand the internal state of the robot,
and program new tasks. However,
how to design an understandable interface for
non-expert users remains to be explored.
To this end, we aim to discover patterns that naturally arise
when ordinary people try to visually communicate the knowledge of a robot
using cards on a canvas.
We describe the process of discovering and formalizing these patterns.
Then, we derive design recommendations by prototyping and Wizard-of-Oz testing.
We began by creating comic strips to represent stories of
knowledge communication between a human and a robot.
We took care to depict the scenario in a way
that does not hint at how the knowledge is presented,
to avoid influencing the study outcome.
We then designed hexagonal cards to represent pieces of robotic knowledge.
The hexagonal design allows for versatile arrangement and efficient use of space.
The color code of the cards corresponds to common types of situational knowledge,
which is summarized from a literature search.
The content of the cards is filled in according to each scenario.
The 12 participants of the study had no prior knowledge of KGs or robotics.
We presented them with the scenario comics,
and asked them to play the role of the robot.
They arranged the cards and drew on a canvas in a way
they think is most understandable and clear.
Then they narrated the canvas using a rectangular frame
meant to represent the limit of the robot screen.
Showing here is a timelapse of the entire process.
To extract common patterns from the final canvases,
we developed a five-step process that consists of
iterative coding and verification.
The coding is done in two levels.
One for local, smaller patterns, or low-level patterns.
One for the overall structure of the canvas,
or high-level patterns.
We used diagrams to formalize them to arrive at the final pattern library.
The low-level patterns, or components, provide various ways of
grouping knowledge elements to communicate short pieces of information —
like using affinity to group similar knowledge,
or using location on the canvas to indicate where events happened within the room.
The high-level patterns, or modules, are ways to combine
and organize components to tell a more complex story.
For example, participants may nest many smaller patterns
within a larger flowchart pattern,
to show the robot's reasoning and knowledge
it used while carrying out a task.
We asked our co-researchers not involved in the analysis
to design a set of knowledge communication interfaces
for a lab service robot, using the pattern library.
We then tested the interface using Wizard-of-Oz techniques on a Temi robot.
During the process, we found that an advantage of using the patterns
is that they can be directly translated to graph database queries.
Meanwhile, designers can design interfaces
without detailed knowledge of the KG structure.
Overall, the 10 participants of the testing
responded positively regarding the interface.
Interviews with them highlighted several design recommendations:
Technical language in the ontology can confuse users,
so it's better to use user-friendly language when presenting KG data.
Loops in the KG structure and a large number of similar elements also confuse users
when browsing the interface.
The classification tree pattern can help ease this problem,
but at the expense of increasing interface depth.
Browsing KG using the entity and relevant information pattern
is easier than browsing by ontology,
but a preview of the neighborhood might be needed
to support this interaction.
If you wish to learn more,
please refer to our paper for more details.
The cards, dataset, photos, and code we used
are available on the project GitHub.
Thank you very much for your attention and interest,
and we wish you a great day :)