An independent randomised controlled trial found that students who used the Medly AI revision platform made significantly more progress in GCSE science than peers who spent the same time on their usual self-directed revision.
Trial design and headline results
The study, evaluated by WhatWorked Education, ran for four weeks with Year 9 and Year 10 pupils in mainstream English secondary schools and tested Medly AI across Physics, Chemistry and Biology. A total of 929 students completed the pre-test, and 644 students completed the post-test. Among those completing the post-test, 332 were assigned to Medly.
Across the combined sample, the Medly group recorded a statistically significant improvement compared with the control group, with an overall effect size of 0.33. The gains were statistically significant in each science individually:
- Biology: effect size 0.52
- Chemistry: effect size 0.32
- Physics: effect size 0.31
| Subject | Effect size |
|---|---|
| Biology | 0.52 |
| Chemistry | 0.32 |
| Physics | 0.31 |
What the comparison entailed
A notable feature of the evaluation was its active control. Both groups were allocated 30 minutes a week to revise the topic being studied over four weeks. Pupils assigned to Medly completed predefined platform activities; the control group used their normal self-directed revision resources, excluding Medly. Teachers who described their control-group resources mentioned examples including Tassomai, Sparx Science, BBC Bitesize and printed materials, though the report notes the teacher response rate for that part of the evaluation was low and not necessarily representative.
Strengths, limitations and interpretation
The trial’s strengths include random assignment, a defined revision dose and separate reporting for each science. The overall effect size of 0.33 indicates a small-to-moderate benefit across the combined sciences; the larger effect in Biology (0.52) suggests subject-specific differences in how the platform supported learning.
However, the evaluation also recorded meaningful attrition: only 644 of 929 pupils who completed the pre-test finished the post-test. The low teacher response rate around the control-group description limits the precision of comparisons about what “business-as-usual” revision looked like in practice. These caveats mean the results should be seen as promising but not definitive evidence that Medly will produce the same gains in all settings.
Implications for South African classrooms
For South African education planners and schools, the study offers several lessons:
- Short, structured doses of curriculum-aligned practice can deliver measurable learning gains compared with undirected revision.
- Edtech impacts may vary by subject; platforms should be evaluated separately for Biology, Chemistry and Physics.
- Context matters: differences in curriculum alignment, device access, teacher support and student background will affect transferability.
While the trial took place in England and measured GCSE outcomes, the core finding—that an adaptive, feedback-rich platform produced larger gains than self-directed revision given the same time—will be of interest to South African schools seeking scalable interventions to strengthen matric and earlier-year science achievement. Any adoption locally would require alignment to the CAPS curriculum, piloting in representative schools, and careful monitoring of access and equity.
In sum, the WhatWorked Education evaluation reports that Medly AI produced statistically significant improvements in GCSE science over a short, controlled trial period. The results warrant further trials in different education systems and longer-term studies to establish whether gains persist and translate into higher exam performance.
