It's my great pleasure to be here, and talk to you about our work on analyzing the privacy
and safety aspects of medical voice applications by vetting policy enforcement.
I'm Faysal from University of Virginia, this is joint work with Hang Hu, from Virginia
Tech, Prof. Gang Wang from University of Illinois Urbana Champaign, and my advisor Prof. Yuan
Tian.
Lots of rumor already going on over the internet, claiming Alexa is not mature enough to provide
medical suggestions.
Some claim that it will degrade future health care system.
While others prove it by performing a small scale analysis on a handful amount of skills.
Amazon refer all the voice applications as Skill.
In their experiment, they asked most commonly used dispensed medications in US and found
that half of the time alexa fails to recognize those medications.
After the deal with NHS, Secretary of State for Health and Social Care of UK Matt Hancock
says it can be fit for the future but not now.
Because, Alexa does not have the sophisticated AI to provide medical suggestion right now.
Moreover, another group of researchers randomly selected a few (16) health skills that provide
a diagnosis of symptoms and treatments.
Majority of the skills failed to provide valid answer.
These all events inspire us to run a large scale analysis.
While building the tools, we experienced with several key challenges.
First, we do not have any executable files for skills neither any source code.
On top of that, all the natural language computation happens in the cloud which is a black box
to us.
So we don’t know the underlying behavior of the skills.
While building the tools, we experienced with several key challenges.
First, we do not have any executable files for skills neither any source code.
On top of that, all the natural language computation happens in the cloud which is a black box
to us.
So we don’t know the underlying behavior of the skills.
We solved the first challenges by utilizing the alexa developer’s testing tool.
Using this tool we can enable/disable skills and communicate with the skills.
to investigate policy violations, we need to generate appropriate testing input which
requires a combination of language modeling with safety and privacy aspects.
Our tool VerHealth is able to perform multiple rounds of interaction.
Our answer generator component is able to generate leading answer which Alexa accepts.
We also rephrase the commands because of lacking functionality of language interpretation on
the Alexa end.
It is very difficult for us to verify the responses from the skills.
Like if skills suggest to take a medicine we don’t know whether it is appropriate
or not.
We seek help to the medical experts.
As they have the domain knowledge they can easily figure out the appropriate/inappropriate
medical suggestions.
With that, we addressed the challenges that we faced during the experiments.
Our tool contains two modules – static and dynamic detection module.
We will check all the policies that requires interaction using dynamic module
whereas if there is no interaction involved then we will use our static detection module.
For checking policy of claiming life saving assistance we use our rule based approach
to find the violated skills and for checking the presences of disclaimer, we used our CNN
model.
Using those two component we build our static module and report on the violated skills.
We built our dynamic module with six major components.
You can read our paper for the implementation details.
We use our tool to detect policy violations of the health related skills.
This is the full results from our experiments on the five different policies.
out of 813 healthcare skills, we detected 244 skills (30.23%) storing user physical
and mental health information(violating HIPAA); we also detected 697 (86.36%) skills missing
therequired disclaimers.
The gap is likely due to the fact that VPAplatforms such as Amazon Alexa lack the necessary vetting
processon healthcare applications.
out of 562,302 total responses collected, the vast majority are template responses from
a small set of medical databases.
The results suggest that most healthcare skills are simply matching the pre-scripted answers
based on user questions but they are struggling to understand users’ true intentions.
on one hand, we need a substantially richer medical knowledge base in the backend.
On the other hand, we need more advanced natural language processing techniques to infer userintention
in order to generate relevant and helpful answers.
the domain experts pointed out that the skills are not qualified to provide emergent support
for mental distresses.
In addition, for more than 50% of the cases, the experts believe the skills should have
asked more questions before giving the advice/diagnosis.
These results suggest a critical problem for future research, that is, how to improve the
context awareness of the healthcare skills.
To balance the need for privacy protection and rich functionality, a path forward is
for VPA applications to facilitate a deep integration with hospital services and healthcare
companies
Categorize the diseases which will require the suggestions from the real doctor.
Because sometimes a patient may have multiple diseases, medicine of one will conflict with
others.
In those cases, make patient connected with a real doctor or at least advice them to contact
with a real doctor is necessary.
Skills should ask Follow-up Questions and Suggest different treatment for different
causes of the same disease.
The experts suggested that some treatment was not appropriate for all the etiologies
of a certain disease.
That’s all I have.
Thank you all for your time for attending my talk.