Title Slide: I am Amna Batool from the University of Michigan, and today I am presenting my
paper on “Detecting Data Falsification.”
Data falsification in healthcare is a well-studied topic that includes intentional and unintentional
fabrication of data in the form of omissions, errors, and approximations.
In our context, it is the intentional fabrication done by the frontline healthcare workers in
the data collected from the field.
To counter this, many LMICS have mid-level managers who are the immediate supervisors
of these frontline workers responsible for monitoring their staff members’ performance.
Despite the vital role these mid-level managers play in ensuring the quality of data collected
from the field, they remained an understudied topic in HCI and CSCW.
Therefore, we conducted a qualitative study with the mid-level managers of Pakistan’s
immunization system.
We interviewed and observed 22 mid-level managers in 4 months in 5 districts of Punjab, Pakistan.
We found that mid-level managers, aka Assistant Superintendent Vaccination or ASVs in Pakistan,
use several strategies to identify falsification from paper-based documents used by frontline
health workers
We divided these strategies into 4 major types, including triangulation, anomaly detection,
supplementary data collection, and interrogation.
Not only these strategies are effective to catch intentional falsification, but they
could also readily be incorporated into the existing digital platforms used by these mid-level
managers and workers to improve the efficiency of the overall system
To brief about the application of these strategies in the field by ASVs, I’ll provide two examples
from the paper along with the design recommendation:
Example 1: While using anomaly detection, ASVs look for patterns that deviate from formal
data entry protocols.
For example, an ASV explained his anomaly detection strategy as: “When I checked his
[vaccinator] daily register, there were six entries each day for the last one and a half
months.
Neither 5 nor 8, he exactly inoculated six injections in routine for the last 1.5 months...[which
is impossible].
This whole data was fake and it means he submitted his monthly report based on this false data..."
This strategy helps ASVs to catch problems related to inflated coverage and can easily
be incorporated into the data collected by vaccinators on their mobile application by
simply comparing the count of daily data points.
Moreover, in case, protocol deviation happens, ASVs can be informed through their mobile
application.
Example 2: While using interrogation techniques, ASVs use their experience to identify any
wrongdoing.
While explaining an interrogation technique under MT, an ASV told us: "Once I was going
for my regular visit when I saw a vaccinator cross me on his bike.
He was supposed to be at his kit station.
I called him on his mobile phone and asked for his location.
He said that he is at the kit station.
I said but my application is showing me your location on this road (the road where I was
standing and he just passed by).
He got afraid and started apologizing to me and said that he had an important task to
complete and he will be at work in an hour."
This strategy helps ASVs to figure out vaccinators’ attendance in the field.
This too can easily be transformed into a digital design by providing working hours’
location of vaccinators to ASVs.
Moreover, notifying ASVs through their mobile application in case vaccinators cross their
location boundaries during working hours.
We have provided such digital recommendations in our paper, which could be easily used in
HCI/CSCW interventions related to frontline healthcare workers.
However, integration of systems working at different levels of hierarchy is required
to make this happened that we have discussed and recommended in our paper.
We have also observed possible patterns on how the managerial style of ASVs affects their
strategies for detecting falsified data.
ASVs who prefer largely supportive supervision tend to use triangulation more often.
In comparison, ASVs who follow a more punitive management style focus on supplementary data
collection and interrogation techniques to find mistakes quickly.
With respect to global development, while health systems vary greatly across countries,
we believe that some elements of Pakistan’s immunization system occur in other developing-world
contexts and that our work highlights themes that could guide work in other contexts.
Thank you!