Eedi trial finds AI tutoring gains on lesson tasks, but not on broader math test

Eedi released expanded findings from its classroom AI-tutoring trial on October 29, reporting a distinction that matters for schools: better performance during lesson tasks did not translate into an advantage on a broader standardized mathematics test during the study. The company's account describes a randomized trial with Google DeepMind covering 1,179 students in 10 UK state schools, with ten weeks of lessons across a twelve-week term.

Students stayed in one of four support conditions: standard static material, expert human tutoring, or one of two AI-tutoring arrangements. The AI was embedded in Eedi's teaching architecture rather than offered as a general chatbot. Human tutors reviewed its draft messages before students received them. The results therefore concern a supervised system, not autonomous AI instruction.

On immediate follow-up questions, tutored students generally did better than those receiving static material, with no clear difference between AI and human tutoring. But students assigned AI tutoring did not outperform the static-content group on Star Maths. Eedi reports that exploratory differences between girls and boys require further validation. They are not a sound basis for assuming that every student in a demographic group will respond the same way.

Google's account proposes that detailed tutoring took more time per problem, leaving students exposed to fewer curriculum topics. That is an interpretation of the mismatch between narrow and broad assessments, rather than proof that a longer trial must eventually produce better test scores. Google says it is exploring longer follow-up studies.

The 74's independent reporting, which includes an interview with Eedi's chief impact officer, adds context about implementation. Only 916 students completed both standardized assessments. The reporting also highlights the cost of expert oversight and the challenge of keeping students engaged in tutoring conversations.

For school leaders, the useful question is which outcome a proposed tool has actually demonstrated under which conditions. Correcting a misconception during a lesson is valuable, but differs from showing durable improvement across mathematics. The supervision model, study duration and missing assessments all matter when interpreting the evidence. These findings offer a reason to examine classroom AI carefully, with measured outcomes and human workload visible, rather than treating one positive exercise metric as proof of a general educational benefit.