This video presents the development and evaluation of an interactive AI system
that supports collaborative decision-making with therapists on rehabilitation assessment.
With advanced machine learning,
algorithmic-assisted decision making is being
considered and explored for various domains in the form of a decision support system.
This work focuses on clinical decision support systems in the health domain.
Such a system has been considered as a promising way that provides clinicians
computational information on the status of a patient to improve their clinical decision
making (e.g. rehabilitation assessment)
However, the adoption of these systems in the practice remains challenges due to
various factors, such as lack of user-centered designs, the opaqueness of AI systems,
and not being robust to new data.
We have limited knowledge on the gains of these systems and
how human-AI can collaborate with each other on a decision-making task.
This work focuses on a decision support system for stroke rehabilitation assessment.
Through iterative engagement with therapists and post-stroke patients,
we create a human-AI collaborative approach for rehabilitation assessment.
This approach integrates machine learning with a rule-based model from therapists to
support human-AI collaborative decision-making. First, this approach applies reinforcement
learning to dynamically identify salient features of assessment and predict the quality of motion.
Given these salient features, this approach generates patient-specific analysis to therapists.
After reviewing patient-specific analysis from AI
therapists can provide feature-based
feedback to a rule-based model for tuning an imperfect model
After creating this approach, we conducted user studies with therapists
to analyze the effect of presenting an analysis of AI on therapist's decision making
and accomodating therapist's feedback on AI
For the first research question, we compared therapists' experiences of
assessing patient's performance using three variants of a system:
First, the traditional that only shows a video of patient's exercises
The second, the predscore that shows videos with only predicted score of AI
the last, the proposed system that shows videos with predicted scores & patient-specific analysis
In addition, we had a follow-up study with 5 therapists
to review patient-specific analysis and collect their feature-based feedback on each patient
such as adding, removing a new feature,
or adjusting the threshold value of a feature-based rule.
Overall, our proposed system achieved higher scores on various perspectives.
Specifically, therapists had higher usage
intent on our system, which provides richer information
and reduces their effort on assessment
while significantly improving their agreement level.
In addition, while accomodating therapist's feature-based feedback,
both rule-based and hybrid models significantly improve
their performance to replicate the therapist's assessment.
Overall, the interactive hybrid model significantly performs better than
the non-interactive neural network and therapists' agreement level.
to conclude, this work highlights the value of simple but user-centered explanations from AI for
effective human-AI collaboration,
Our results show that both experts and AI can learn from each other's
strengths for improving decision-making.
For example, the user-centered analysis of AI provides new insights to experts
and supports them to achieve more consistent assessment with less effort.
After interacting with AI, an expert can understand
the capabilities and limitations of the AI system
and provide feedback on the AI system for more accurate predictions.
Thank you