hi my name is sam robertson and today
i'm presenting the paper
modeling assumptions clash with the real
world transparency
equity and community challenges for
student assignment algorithms
which i wrote with co-authors tonya noen
and nila farselahi
many public school districts use
matching algorithms to assign students
to schools
these algorithms have promising
theoretical properties but districts
have encountered challenges in practice
in this study we explore why san
francisco unified school district's
algorithm
has not supported their goals of
transparency equity and community
we do this by comparing theoretical
modeling assumptions to families goals
and constraints in the real world
in this analysis we took a
value-sensitive approach and combined
policy analysis
13 interviews with parents and
quantitative analysis of kindergarten
application data
in this talk i will start with a very
quick overview of how these systems work
then i will summarize the challenges we
found with respect to two of the
district's goals
transparency and equity i will do this
by briefly explaining the theoretical
properties that made the algorithm
seem like a promising solution and then
contrast these to what happened in the
real world
you can check out our paper to read more
about the district's other goals
promoting quality schools and a sense of
community
i will end the talk by outlining the
implications of our work for the design
of student assignment assignment
algorithms
and high-stakes algorithmic systems more
broadly in districts with school choice
programs students have to apply to the
schools that they're interested in
usually demand for schools is unevenly
distributed and
not every student can attend their first
choice school because of capacity
constraints
student assignment algorithms give
school districts an efficient way of
determining
how to assign students to schools these
algorithms ask families to submit a
ranked list of schools they would like
to attend
schools can also define priority
categories for certain students
for example siblings of current students
researchers in the field of mechanism
design have developed algorithms that
find a good matching between students
and schools
accounting for both sides preferences
but what is a good matching
in the next part of the talk i'll walk
through some of the properties of these
algorithms that make them an attractive
solution to the student assignment
problem
and how these properties have been
challenged by real-world conditions
the district's first goal was to ensure
that their assignment
system was transparent and easy to use
matching algorithms are often considered
simple and transparent because they
follow a clearly defined and explicit
procedure
plus they are provably strategy proof
meaning parents do not benefit from
manipulating their preferences
to try to improve their outcome however
parents still found the system difficult
to understand
and some believe that parents with
privilege could gain the system to their
advantage
further the main information that they
needed was information about the schools
not information about the algorithm and
they found it difficult to find and
navigate relevant school information
the second key goal for the school
district was to promote racially and
socio-economically integrated schools
and to promote equitable access to
high-quality education
the district hoped that the system would
promote diversity and equity because it
allowed all students to apply to any
school in the district
further the district specified that the
algorithm gives students living in
areas of the city with underperforming
schools priority access to their schools
of choice
however less than a decade after the
system was introduced
diversity in schools had actually
decreased and some schools were less of
us
than the neighborhoods they were in we
found that part of the reason for this
is that preferences remain segregated
across the city
for example because of transportation
costs to schools photo from home
the algorithm is optimized to satisfy
preferences so it's unable to create
devos assignments if the applicant pool
at each school is not
bus based on our findings we present
recommendations for student assignment
algorithms
which can be also relevant to other
high-stakes algorithmic systems
for example one problem in san francisco
is that the algorithm is optimized to
satisfy students preferences
but this objective doesn't align with
distributive goals like classroom
diversity
this suggests that there is a need to
realign algorithmic objectives with
community goals in mind
another problem is that families have to
do a lot of work to figure out how to
express their needs and goals
in the form of a ranked list of schools
one path forward could be to reconsider
how families express their preferences
to the district
this opens opportunities not only to
make the process easier for families
but for the district to reduce the
emphasis on individual outcomes
and encourage community members to think
about distributive outcomes like
integration
if you'd like to learn more about this
work we hope you'll read the paper or
come to one of our q a sessions
thank you