Hello.
My name is Tya Chuanromanee.
And today, I will be talking about our paper, Supporting
Storytelling with Evidence in Holistic Review Processes--
A Participatory Design Approach.
We performed some participatory design workshops
with some college admissions officers
and examined how the college admissions process
is a sensemaking process.
So do you remember when you were a senior
in high school applying for undergraduate programs?
There was a lot you had to keep track of,
including taking the SAT and the ACT, maintaining your GPA,
and getting extracurricular experience.
There was also the matter of which schools you applied
to, and then after that, waiting for months
to find out whether you got into the school that you wanted.
So in our paper, we took a look behind the scenes
of what happens during those long months of waiting.
We investigate how readers or college admissions officers,
approach a holistic college admissions
process, with a special focus on committee review.
There were eight committee reviews.
And sure, applications can fall on a spectrum
of definite accepts to definite rejects.
For those that may be more ambiguous,
readers may need to get together to discuss them and come
to a decision.
Here is a mockup version of a decision sheet,
which is an important tool in the process and the current way
that the application is presented to the reader.
It can contain qualitative information
about the student, such at demographics and high school
information, and also quantitative information
about the applicant, such as the GPA and the test score.
It will also include the reader's comments and takeaways
on the application, as well as internal information
such as codes.
So briefly, the process of leading up to the committee
review are as follows.
So first, the reader will go through the application
and draw out the decision sheet, making notes of what
they write in the application.
Each reader is responsible for multiple applications.
So they must refresh the memory of the student application
before the committee review meeting,
which can happen anywhere from the next day to several weeks
out.
Each meeting has a leader, and readers take turns
in sending the student's story and making
a case for the student.
Then the readers will discuss whether or not
the student should be accepted.
As we learned more about the process,
we conducted three participatory design workshops
in the summer of 2019, with seven admissions officers.
Our goal in the first workshop was
to figure out what the challenges and opportunities
are in this domain.
Then we facilitated a photo narrative activity
in a second workshop to learn more about
how participants constructed the story for the community review.
Finally, in the third workshop, we
collaborated the redesigned decision sheet
to better support the process.
For the first workshop, laid the groundwork
of what we planned to do, as we were
asked to get a more concrete understanding of the process.
We also needed support of a narrative activity, which
entailed them looking through an application
and taking pictures or screenshots of whatever
evidence that they would use for committee meeting.
The second workshop mainly involved
the photo-narrative activity.
We asked participants to create a collage
from the screenshot they took between the first and second
workshops.
Reconstructed stories should tell about the applicant
in the committee meeting.
So here is an example of a collage
one of the participants created in this workshop.
We coded all seven and analyzed them with respect
to the narrative structures and genres, introduced
in [?] seminal paper, "Narrated Visualization,
Telling Stories With Data."
Here is the stylized version of a participant's collage.
The photo narrative contains elements of the partition
poster, as well as the comic strip, given the commentary
caption turn in red.
This collage represents a clear flow shot,
with multiple bi-directional path.
The reviewer described it as such,
allowing for [?] for multiple ways
and to circle back if needed.
Our findings inform our approach for redesigning the decision
sheet.
In the third workshop, we worked with readers
to examine the current efficient sheet more closely
and consider possible designs.
This figure shows one potential redesign of the decision sheet,
based on the way the reader structured the story.
I invite you to read the paper for more details on this
and how to design the [?].
So in addition to the redesign, we
identified three important takeaways for the workshops.
So first, review processes are sensemaking processes.
Therefore, evidence collection and storytelling is important.
Second, storytelling is not confined just
to the committee review process, but also
includes forming and supporting arguments
while reading and reviewing information.
And finally, because this is a holistic process,
the goal is for each application to get an equal review.
Thus, the rubric dominates this process, and therefore,
the story structure.
So thank you for listening to my presentation.
Again my name is Tee Chuanromanee,
and here is my contact information.