Informing Age-appropriate AI
examining principles and practices of AI for children.
AI systems are becoming increasingly pervasive within children’s everyday activities,
and they are starting to play a variety of roles in the digital ecosystems of children.
Meanwhile, this use of AI comes with new kinds of risks,
some of which were of particular concern for children.
Yet, understanding the ways that AI-systems are being used for children,
and how can we develop better AI design for children
is still a new and emerging area of investigation.
In order to answer these questions,
we investigated the intersection of the two areas.
We first conducted a policy review,
in which we looked at what are the AI design principles that were promoted by the regulatory community.
We then conducted a landscape analysis of existing AI systems developed for children.
to achieve a mapping between the regulatory efforts and current practices.
Starting with the policy review:
On one hand, there exists a growing body of general regulatory AI frameworks,
but not specifically focusing on how that relates to children.
On the other hand, there is a complementary body of literature focusing on designing age-appropriate digital technologies specifically for children,
but not focusing on the AI aspect.
The two branches of work were separate but related,
and sometimes they touched on similar topics but in different ways.
There is therefore this confusing smorgasbord of different frameworks and guidelines relating to somewhat overlapping concerns.
This can make it difficult for designers and practitioners to effectively establish concrete design suggestions and standards.
We compared these two sets of frameworks and policies,
and conducted a ground-up thematic analysis of key design principles discussed in them.
We grouped the design principles by their specific definitions and identified 10 areas of alignment.
To see how these principles relate to existing AI systems for children,
we conducted a landscape analysis through a systematic literature review
including 188 papers on AI systems developed for children from the HCI community,
constructing a high-level overview of key systems and their characteristics,
including their target audience, kinds of data processed,
as well as application domains and computational methods used.
We further cross-compared our policy review and landscape analysis,
with the aim of identifying whether, and how
the previously identified design principles were addressed in existing AI systems developed for children.
Specifically, we looked at if, and how each design principle is considered through application domains of AI for children
In general, we found that papers from each application domain focused on
a small number of specific concerns related to a small number of principles that were most relevant in each application domain.
Take the Medical Diagnosis systems developed for children for example,
we can see that only a small number of design principles were addressed very locally on some specific concerns.
In fact, the only two principles that were seen as important for this domain are accountability and safety/do not harm.
Meanwhile, when identifying the how
- in terms of how AI systems managed to follow a specific design principle.
We found that some work were grounded in, or guided by theoretical and empirical work.
Such as the use of self-determination theory in some tutoring systems.
However, such expertise was localised to a few domains.
such as personalised tutoring.
The remaining application domains had much less supporting theory available.
Our results showed that we may require new thinking in the area of AI for children.
There are at least two reasons why we think that a unified Code for Age-Appropriate AI Design would be beneficial.
First, our results showed that while designers and researchers did address some of the common principles in their design,
these considerations were localised to a small subset of principles we identified.
A broad, Age-Appropriate AI design code may thus not only serve as a reminder of the other principles,
but to highlight risks and issues that may be no less important than the ones that are most immediately relevant.
The second advantage of establishing a unified Age Appropriate Design Code for AI is that
it may create a space for researchers to assemble expertise on the ways different kinds of AI-related harms affect children differently from adults,
and a corresponding body of theory and empirical findings for mitigation.
What might our analysis suggest about assembling a code for Age-Appropriate AI Design then?
Based on our findings, we synthesised the previously identified 10 common design principles
into five by grouping related considerations together.
For each principle, we highlighted its
core considerations
and relevant challenges for future designers.
Starting with the first, Fairness, equality, inclusion and access.
This principle refers to, not only to discovering the needs of diverse groups
but also ensuring that all children are treated fairly and equally.
Some key challenges we identified include the difficulty in directly involving children in the design process,
and it may be particularly hard for some domain-specific applications consider whether discrimination exists.
The second, Transparency and accountability,
which refers to Identifying a chain of accountability for system mis-behaviour and make systems easy to understand and predict.
In terms of the gaps and challenges,
we encourage future designers to think of new ways for establishing accountability mechanism,
and to take into consideration of the different capability of Individual children on interpreting transparency messages.
The third, privacy and manipulation and exploitation.
To protect children’s privacy online refers to respecting their interpersonal and data privacy online,
as well as taking precautions to forbid the use of their data to make exploitation on them.
On the other hand, sometimes the use of children’s data and subsequent manipulation were actually essential for the functioning of the system,
and it could be hard to determine what counts as good or detrimental nudging and manipulation.
Next, safety and safeguarding,
The word ‘harm’ is centred to safety and safeguarding,
that is to ensure systems do not directly or indirectly endanger children through intentional or unintentional design or behaviour.
Meanwhile, we encourage future designers and researchers to think of new ways to evaluate the safety of systems,
apart from pure technical means,
and come up with more comprehensive definition of what indeed counts as harmful content for children.
Finally, sustainability and age-appropriateness,
which is around children’s long-term well-being and development.
Gaps we identified include that the theoretical and empirical groundings for this principle were relatively thin,
and the current evaluation criteria used for assessing whether an AI system is indeed sustainable or age-appropriate, is sometimes vague.
The contributions of this paper are three-fold:
The first, a clarification of the overlaps among the leading regulatory efforts for AI for children.
The second, a landscape analysis of existing AI systems developed for children,
and finally, a synthesised framework to inform a new “Code for Age-Appropriate AI”,
as well as the gaps and challenges that future designers could think about.
We hope this work would provide timely input to emerging policies and standards,
and present an opportunity to ensure a multi-dimensional safeguarding of children in future AI systems.