Retrieval-Augmented Tutoring: Effects on Undergraduate Learning Outcomes

Submitted to LAK '26 · Track 3, Learning Analytics in Practice

Abstract

Evidence that LLM tutors improve measurable learning outcomes remains thin (Kasneci et al., 2023). Across a semester-long trial of 412 undergraduates, students assigned to a retrieval-augmented tutor gained 0.31 SD on end-of-term assessment (Zhang & Okonkwo, 2024).

Effects were strongest among students entering with below-median prior attainment, a pattern consistent with earlier work on adaptive scaffolding (Vygotsky, 1978)(Doo et al., 2020).

Gains did not transfer to a delayed retention test administered six weeks later, which we discuss as a limitation of session-bounded tutoring.

Keywords: intelligent tutoring, retrieval-augmented generation, learning analytics, higher education

1. Introduction

Large language model tutors have moved from demonstration to deployment across higher education, often ahead of the evidence base. Institutions now face procurement decisions with little comparative data on attainment, and most published trials run for a single session rather than a full term.

This study reports a semester-long randomised trial across four undergraduate cohorts, comparing a retrieval-augmented tutor grounded in course materials against an unmodified baseline model.