Preparing videos for communicating activities efficiently requires much effort, whether
it be in research, sports or surgery.
Particularly in the field of surgery, time is a valuable resource, a surgeon's schedule
is filled with daily activities including discussing cases with colleagues, making rounds,
patient consultations, and of course operating on patients.
Despite their taxed schedule, surgeons carve out time to record videos of their surgeries
and create summaries.
My name is Ignacio Avellino, and in this video, I will talk about why surgeons summarize videos,
how they do it, and how they collaborate to divide the workload.
Trigger warning!
This presentation includes videos of actual surgery.
We applied a filter to the videos to reduce their visual impact, and we will show a warning
in advance like the one at the bottom of the screen.
Video is at the center of Minimally-Invasive Surgery.
Surgeons insert an endoscope, a camera, into the patient along with tools they manipulate
while looking at the endoscopic video on a monitor.
Recording the activity is straightforward, as surgeons simply need to record the endoscopic
video.
However, finding relevant moments for communicating the activity is inefficient, as surgeons need
to seek through a great deal of routine activities.
Therefore, surgeons summarize their videos, condensing the important moments in 5 to 10
minutes of footage.
Now, although there are automatic approaches for summarizing videos that make use of features
such as audio level, changes in scene angle, colors, and motion, these however are not
adequate, because surgical videos do not typically include audio, they are fundamentally a single
long shot that lasts hours, and the colors are rather uniform.
Therefore, surgeons manually summarize videos.
But, as this requires much time, they summarize fewer videos than what they would like to.
To inquire about surgeons' practices surrounding video, we conducted semi-structured interviews,
we observed surgeries, we observed how surgeons summarize video, and carried out a survey
to validate our findings.
The results show, first, how surgeons use summarized videos, which highlights their
importance; second, how surgeons summarize videos, which highlights the high costs this
process incurs; and third, how surgeons collaborate to summarize videos.
Let's look at these in detail.
We identify six uses of summarized videos.
The first use is to improve one's own practice.
Surgeons review video summaries of past surgeries as part of their preparation routine, before
performing an unusual surgery.
The second use is for education.
Participants reported using videos to teach strategies, show different ways to perform
a particular gesture, including what to avoid, or to illustrate complications that can be
used to trace mistakes.
The third use is for discussing cases.
Videos can objectively convey the extent of diseases, like cancer, when presenting their
severity.
This can make them more appropriate than other materials, such as subjective scores, for
making decisions.
Videos can also be used for scientific conferences and studies.
Conference calls can have a specific focus for videos, which include rare procedures,
particular anatomies, rare complications, or even compilations of a surgeon's worst
moments.
The fifth use is for communicating with the public, such as patients, or wider audiences.
Lastly, for keeping legal traces of their surgeries.
In our followup survey, 31 participants rated the usefulness of each of the six cases on
a 7-point scale.
Most survey respondents agreed that education and scientific publications are the most useful
ones.
The diversity of uses reflects that videos are an important tool for surgeons.
But what is the cost to produce these videos?
When we asked how much time it takes to summarize a video with respect to its duration, respondents
indicated that it almost always demands more time than the surgery itself, even up to six
times.
This leads us to our second main finding: how surgeons summarize videos.
In our study, participants started with the full video and iteratively removed video sequences
until reaching a desired length.
We formalize this process in five steps.
The first is Overviewing, seeking through the full video to get a sense of the surgery.
Then, Cleaning, which consists of removing irrelevant segments where nothing useful is
visible, like when the endoscope lens is dirty or outside the patient.
The third step is Coarse Sequencing, where surgeons identify the broad strokes of a surgery,
which usually correspond to the procedure stages.
Then Fine Sequencing, where surgeons trim long sequences that show multiple repetitions
of the same gesture, selecting one execution that best exemplifies it.
And finally Contracting, which consists of removing uninformative frames that are considered
"lost time" within a sequence, to show only the beginning and then end of the process.
These steps tell a story of why the extensive time devoted to summarization is not a conscious
choice, but rather the result of inadequate technological support.
Now, one shortcoming is that consumer video editing tools impose a working paradigm, as
they require users to remove video segments.
However, participants repeatedly showed their intention to retain segments.
Thus, we posit that summarization would consume significantly less time if tools would support
selecting sequences to keep, instead of those to remove.
The other shortcoming is that video editing tools come into action after video is recorded,
and thus semantic information that arises during surgery is not captured.
To explore the utility of this information, we encouraged participants during our second
round of surgery observations to indicate key moments that could be relevant for summarizing
the video.
Participants showed great interest in capturing moments that relate to the surgical steps,
the patient's anatomy, particular surgical gestures and unexpected or critical moments.
Our third and last main finding relates to the collaborative facet of the summarization
process.
Participants performed video summarization together with colleagues mainly for efficiency,
of course to divide the time-consuming task.
This collaboration happens asynchronously, as one person starts the process, and hands
off an unfinished summarization project to another person.
A common task division we observed was surgeons first delegating the initial steps of Cleaning
and Coarse Sequencing to a resident, who produces a shorter video so that the surgeon can then
perform the steps of Fine Sequencing and Contracting.
To collaborate, participants did not rely on functionalities within video-editing tools,
simply because they do not exist.
Instead, they developed ad-hoc coordination strategies.
Aiming to make knowledge persistent as a form of articulation work.
We identify three strategies.
The first is segmenting the live recording during surgery.
In our study, some surgeons asked a nurse to stop and restart the recording by pushing
the record button.
This strategy can serve two purposes: either excluding sequences that surgeons know are
not interesting, or, creating multiple files to be used as navigational landmarks during
summarization.
The second ad-hoc coordination strategy is to capture moments on external media, for
example taking pictures with a mobile phone as a reminder of events to keep in the summary.
The third strategy takes place after surgery, when summarizing video.
It consists of engraving marks on partial video summaries, to either signal that a sequence
is important, or to coordinate the process, for instance that a sequence is finished and
that the next one needs editing.
These ad-hoc coordination strategies have their own drawbacks, as stopping the recording
can lead to the loss of important moments, and marks such as gaps in the live video,
photos on external media, or title cards in the summary, not only require scanning the
full video to find them, but most importantly they do not carry semantic meaning, making
it difficult to use them for coordination.
Now, before wrapping up the talk, I want to emphasise a key takeaway: consumer video editing
tools do not support the articulation work that is needed to coordinate interdependent
tasks.
This leads to breakdowns in coordination, specifically, in the division of tasks and
assignments, (who does which step?), and, in the management of dependencies, (is this
step done so that I can start the next one?)
We posit that two key aspects are missing.
First, enabling the capture of information during the data-generation stage, and transferring
it to the data-processing stage.
Commercial cameras such as the GoPro, have started to implement functions to leave marks
on videos while they are recorded.
This is only a first step, as we need mechanisms to include a layer of semantic information
that gives meaning to the marks.
Second, making the common field object, in our case the video itself, also a coordination
artifact.
Remember that a coordination artifact is a symbolic object used for coordinating actions.
For example, a shared checklist where, as multiple people check off items when they
complete them, they can coordinate action.
One way to make the common field object a coordination artifact, is to use the same
layer of semantic information to render explicit protocols, conventions and the state of processes.
In our case, this would mean coupling the video footage with information about which
segments need to be kept, who should perform the different steps, or which segment needs
revision.
Now, to wrap up, let's go back to summarization itself.
Throughout our study, surgeons expressed their desire for automatic methods.
But these are unlikely to achieve the desired outcome, as nuances like the purpose of the
video or the intention of what to communicate are difficult to model.
We believe that a compromise between automatic and manual methods is a much more promising
approach.
Surgeons would capture semantic information during surgery that a system uses to work
on the different summarization steps, proposing paths, alternatives, which then the surgeon
can work with.
That is all for this talk, on behalf of my colleagues thank you for listening.
And please do reach out to us with your thoughts.
And of course we hope to see you at CSCW!