Artificial intelligence offers new ways to support children and adolescents with mental health needs, but current systems are not yet ready for routine use in schools or clinics without careful attention to fairness, inclusivity and bias, say researchers from University College London (UCL) and the Children’s Hospital Medical Center.
Complexity of childhood expression challenges AI
In a recent commentary, Julia Ive and Paulina Bondaronek of UCL and John Pestian of the Children’s Hospital Medical Center argue that the characteristics of young people’s development make automated assessment and support difficult. They highlight that children and adolescents often have trouble articulating their inner experiences, and that their language, behaviour and symptoms can vary widely by age, sex and other sociodemographic factors.
The authors note several specific hurdles that any AI using natural language processing (NLP) must overcome before being considered safe and effective for young users:
- Developmental variability: expressions of distress differ greatly across age groups and can change quickly as a child develops.
- Cultural and family context: diverse cultural backgrounds and family norms affect how emotions are described and interpreted.
- Access and consent: children frequently cannot access care independently, relying on adults to recognise problems and enable follow‑up.
- Data limitations: training datasets may under‑represent certain groups, leading to biased or inaccurate outputs.
Fairness and bias are central, not optional
The commentary emphasises that fairness and bias are not peripheral technical issues but core clinical and ethical concerns. If an AI system systematically misinterprets how a particular group of children expresses distress, it could worsen inequalities in care rather than reduce them.
To address these risks, the researchers call for multi‑disciplinary approaches that combine clinical expertise, developmental psychology, cultural competence and technical safeguards. They stress that progress requires rigorous evaluation of AI tools on datasets that reflect the full diversity of children and adolescents who might use them.
Implications for South African schools and services
South African educators, school counsellors and provincial health services should therefore treat AI‑based mental‑health products with caution. Enthusiasm for technological solutions must be balanced with strong governance, transparent validation and continuous monitoring. Practical steps recommended include:
- Insisting on external validation studies that report performance by age, sex, language and population subgroup.
- Maintaining human oversight: AI should augment, not replace, trained clinicians and school counsellors.
- Embedding cultural and contextual knowledge into model development and deployment.
A useful way to see the immediate priorities is in how systems perform across key domains:
| Domain | Priority |
|---|---|
| Clinical validity | Demonstrate accurate detection across developmental stages |
| Equity | Assess and mitigate differential performance by sociodemographics |
| Access | Ensure pathways to care for children who cannot self‑refer |
The researchers caution that without these safeguards, deployments could produce false reassurance, missed diagnoses or inappropriate interventions — outcomes that would be particularly harmful for young, vulnerable users.
While AI has the potential to provide timely and scalable support, the authors conclude it must be developed and implemented with the same rigour applied to clinical tools. For education stakeholders in South Africa, this means prioritising validated, culturally aware systems and keeping learners’ safety and equity at the forefront of any adoption decision.
