10% Better Exam Scores with AI Tutoring — What Our Pilot Study Found (and What It Didn't)
Published: May 10, 2026 | By: Roman Vasylyshyn, founder of Lambdio | Reading time: 8 min
In a pilot study conducted at a university economics course, students who actively used Lambdio's AI-powered spaced repetition platform scored between 9.5% and 11.6% higher on their final exam than those who didn't — despite none of them reaching even 10% of the recommended study time on the platform.
We're sharing the full data here — including the limitations. The sample was small, and the results don't yet meet the bar for statistical significance. But the pattern is consistent enough that we believe it's worth discussing openly.
Table of Contents
- What Is Spaced Repetition?
- The Experiment Setup
- The Results: Group Comparison
- The Dose-Response Relationship
- What This Study Can't Tell Us Yet
- What Comes Next
- Try Lambdio for Free
What Is Spaced Repetition?
Spaced repetition is a learning technique that schedules review sessions at increasing intervals over time — just when you're about to forget something, the system brings it back. This method exploits the spacing effect, one of the most robust findings in cognitive psychology: information reviewed at spaced intervals is retained far longer than information crammed in a single session.
Modern implementations use algorithms to predict exactly when a memory is likely to fade. Lambdio applies this method through an AI tutor that adapts to each student's pace, scheduling reviews based on individual performance rather than fixed calendar dates.
The Experiment Setup
We partnered with The National University of Ostroh Academy to test with economics students whether Lambdio's AI-powered spaced repetition could improve exam performance. Here's how the study was designed:
| Metric | Value |
|---|---|
| Total participants | 30 students |
| Lambdio access group | 13 students |
| Control group (no access) | 17 students |
| Active users (used Lambdio at least once) | 7 of 13 |
| Engaged users (30+ minutes of study) | 5 of 13 |
| Final test | 26-point test (20 ABCD tests, 2 open-ended questions) |
| Recommended study time | ~30 hours |
| Maximum study time achieved | 154 minutes |
Important transparency note: The estimated "full benefit" threshold was approximately 30 hours of study on the platform. The most dedicated student studied only 154 minutes — that's less than 10% of the recommended time. We want to be upfront — this was a small pilot, not a randomized controlled trial.
The Results: Group Comparison
To analyze the data, we divided participants into four groups based on their level of engagement with Lambdio:
| Group | Who they are | Avg. Score | vs. Control |
|---|---|---|---|
| G0 — Control | No access to Lambdio | 20.73 | — |
| G1 — Access Given | Got access, mixed usage | 21.18 | +2.2% |
| G2 — Active Users | Used Lambdio at least once | 23.20 | +11.9% |
| G3 — Engaged Users | Studied 30+ confirmed minutes | 23.75 | +14.6% |
The pattern is clear: the more students engaged with Lambdio, the higher their scores. Even the "Access Given" group — which includes students who barely used the platform — showed a small positive difference.
What About Effect Size?
In educational research, an effect size (Cohen's d) above 0.40 is considered practically meaningful — it means the intervention makes a real difference in a classroom setting. Our most engaged group (G3) showed d = 0.42, which is notable given that none of them came close to the full recommended study time.
To put that in perspective: a Cohen's d of 0.42 means the average student in the engaged group scored higher than approximately 66% of students in the control group.
The Dose-Response Relationship
Perhaps the most telling finding wasn't the group averages — it was the dose-response pattern.
Students who spent more time on Lambdio consistently scored higher. The correlation between study time and test score was ρ = 0.49 — approaching what researchers classify as a strong relationship.
This matters because it's hard to explain a dose-response pattern with confounders alone. If the results were purely due to smarter students choosing to use Lambdio more, you'd expect a weaker, noisier relationship. Instead, we see a clear upward trend: more time on the platform, higher exam scores.
In medical research, a dose-response relationship is considered strong evidence for causality. The same logic applies here.
What This Study Can't Tell Us Yet
We believe in radical transparency about our methodology. Here are the limitations you should consider when interpreting these results:
1. Small Sample Size
With only 4–11 students per group, the results don't reach statistical significance. You'd need approximately 30 students per group to detect a medium-sized effect with confidence. We're planning exactly that for our next study.
2. Self-Selection Bias
Students who chose to use Lambdio more had slightly higher GPAs beforehand. This means some of the score difference may reflect pre-existing academic differences rather than the platform's effect. We controlled for this where possible, but self-selection bias remains a real concern.
3. No One Hit the Recommended Study Time
The target was ~30 hours of study. The maximum any student achieved was 154 minutes — about 8.5% of the target. The results we're reporting come from students who barely scratched the surface of what the platform offers. This is encouraging (imagine what full engagement could achieve), but it also means we can't yet estimate the platform's full potential.
4. English Proficiency Imbalance
The selected group had higher CEFR English scores on average. However, English proficiency turned out to have negligible correlation with test results (ρ = -0.049), suggesting this imbalance likely didn't affect the outcomes.
What Comes Next
We're planning a second experiment with:
- At least 30 students per group (control and treatment)
- Controlled recruitment to minimize self-selection bias
- A structured minimum study time requirement for the treatment group
- Pre- and post-testing to measure knowledge gains directly
We'll publish those results here too — good or bad. If the platform doesn't work in a more rigorous setting, we'll say so. That's the only way to build something truly useful for students.
If you're a researcher interested in collaborating on a larger study, reach out to us.
Try Lambdio for Free
If you're a university student studying economics, psychology, physics, or any of the courses available on Lambdio, you can try it for free. The experiment data suggests even a few sessions may be worth your time — especially before exams.
This study was conducted in collaboration with The National University of Ostroh Academy. We thank the participating students and faculty for their cooperation.