Hello everyone.
Thank you for attending this presentation.
I am Shruthi Hiremath, a PhD student at Georgia tech, and I will be talking about our work
on Deriving Effective Human Activity Recognition Systems Through Objective Task Complexity
As an alternative to direct observation based studies practitioners are now considering
sensor-based human activity recognition systems for automated assessments scenarios.
One such example is to observe individuals with Parkinson's Disease.
This is similar to a clinical assessment scenario where individuals note down they're identified
motor movements as diary entries
Stakeholders in this application scenario include clinicians and health practitioners
who are interested in observing the progression of disease states
In order to build HAR solutions, data analysis pipelines are proposed.
These pipelines detect relevant disease states.
Data is collected from on-body sensors for such individuals and specific motor movements
are detected, which are then compared to self-reported diary entries.
Resources are required to build HAR solutions in terms of data collected, annotations for
behaviors of relevance and researchers who build tools and methods for analyzing such
assessments scenarios.
Promising results obtained from pilot studies have motivated practitioners to use existing
off-the-shelf, HAR solutions for various deployment scenarios.
However, each HAR deployment scenario is unique and thus requires domain specific
modifications for solutions to be usable and effective .
Here we illustrate two possible outcomes scenarios for practitioner based HAR deployments.
In both the scenarios, the performance of the hypothetical recognition system is shown
as a function of the resources that are required in order to reach the expected performance.
In the first scenario, the expected performance as specified by the practitioner can eventually
be reached.
Yet, it is not clear a-priori, how many resources would be required in order to reach the expected
performance.
This is less than ideal for the practitioner who would want to ensure that they do not
run the risk of utilizing all the resources such that none of those would be available
for actual deployment purposes.
Thus reliable estimates are required ahead of time at time ‘t’ as shown in the graph
to ensure that the deployment process is a reasonable one.
These estimates also inform practitioners of whether or not to invest in a deployment
scenario and whether the number of resources required for a certain deployment scenario
are reasonable.
In the second scenario, even with unlimited resources, the expected HAR performance cannot
be reached.
This may be owing to the fact that the assessment scenario is very challenging or the fact that
unrealistic expectations have been set on the expected performance.
Similar to scenario 1, we will need to inform practitioners ahead of time, to inform whether
the deployment scenario is feasible or not, and of if and how to pursue it.
In order to aid practitioners, we provide an objective task complexity assessment framework
that informs practitioners of the challenges associated with the task and provides performance
estimates before actually solving the task.
We do so in a systematic approach where we look into the complexity of a task across
various dimensions and provide guidelines for HAR systems.
The first stage of the framework looks at the characterization of the HAR task, where
we map a task that is comprising of sample data and annotations under onto an objective criteria
of complexity space.
We use raw data collected from movement sensors and categories of complexity measures that
span over three groups.
These categories are related to measures of feature overlap, measures of linear separability
and measures of mixture identifiability.
The complexity measures are then normalized and in the range (0,1) and are visualized
using radar plots.
In the second stage of the framework, we look into the categorization of HAR tasks where
we map 23 different tasks onto the objective criteria of the complexity space.
This is followed by a PCA based dimensionality reduction technique in order to remove any
redundancies that may occur due to the use of multiple complexity measures.
A similarity mapping along with the expert knowledge of the underlying tasks is then
used to provide a categorization for HAR tasks.
We obtained six clusters from this categorization.
Cluster one represents datasets, which look into specific target behaviors. Two for datasets,
which have activities collected from off body sensors, three for activities involving arm
gestures, four for activities which are repetitive in nature, five for simulated activities and
six for activities collected from device embedded sensors.
In the last and final stage of the framework, we look at the deployment for a novel task.
The novel task is mapped onto the complexity space using the task complexity analysis.
This is then mapped on to one of the clusters identified using the categorization of
HAR tasks.
This mapping provides guidelines for the novel task, as well as recognition estimates before
actually solving the task.
We perform a leave one task out cross validation approach, and we obtain three categories of
results.
The first data set is representative of tasks with the optimized configuration performs
substantially better than the baseline configuration informing practitioners of realistic recognition
performances that can be obtained for such deployment scenarios.
The second data set is representative of tasks where the optimized configuration is not impressive.
Hence this informs practitioners that existing state-of-the-art techniques for such tasks
are not adequate and substantial efforts will be required in order to build analysis pipeline
for such tasks.
The third data set is representative of tasks, where there are no existing state of the art
analysis techniques to derive analysis pipelines from, thus this informs practitioners that
for such tasks, analysis pipelines would need to be built from scratch.
Thus to conclude we have in this work proposed a framework that characterizes a task using
complexity measures, categorizes various tasks, using a similarity mapping and provides guidelines
as well as recognition estimates on a novel task before actually solving the dusk.
Please refer to our paper for more details
Thank you