Practice Makes Progress: What New AI Tutoring Research Says About Rehearsal

Skill is built one rep at a time

Think about the last time a student finally "got" long division, a thesis statement, or balancing an equation. It almost never happened during the explanation. It happened on the fifth or sixth attempt, after a wrong turn, a quick correction, and another try.

We know this intuitively, and the learning sciences back it up: skill comes from repeated, effortful practice with timely feedback. The hard part has never been knowing that. It's providing it. With 25 or 30 students in the room, we can't sit beside each one, watch every step, and catch every misconception the moment it appears.

Why earlier tutors fell short

Computer tutors have chased this goal for decades. Rule-based "cognitive tutors" from the 1990s could check each step of an algebra problem and offer a hint, and the best of them came close to human tutoring (VanLehn, 2011). But they could only recognize the mistakes their designers had programmed in ahead of time. A misconception nobody anticipated, or a student explaining their confusion in their own words, fell outside what the system could see. Building even one narrow unit took enormous effort.

The first wave of chatbots had the opposite problem. They could talk about anything, but by default they handed over answers. In a high school math study (Bastani et al., 2025), students using unguarded GPT-4 did 48% better on practice problems. Then they scored 17% worse than peers once the AI was taken away. The AI did the reps. The students didn't.

What the new research shows

A study released last month points to a third path. In StudentBench, about 2,400 adults spent one hour on GRE practice with an AI tutor, an expert human tutor, or no tutor. Each AI tutor saw the student's pre-test mistakes, built a short lesson around them, generated fresh practice problems targeting those errors, and coached the student through them in conversation. Across 13 AI tutors, learning gains were statistically equivalent to those from expert human tutors.

The most telling finding is why it worked. Faster AI replies went with more student messages. More messages went with more correctly solved practice problems, and more correct practice went with bigger gains. In other words, the AI's value showed up as more reps, each one aimed at a specific misconception.

This echoes a 2025 Harvard randomized trial (Kestin et al.), where an AI physics tutor built on research-based teaching principles produced median learning gains more than double those of a strong active-learning class. In the Bastani study above, a tutor version designed to give hints instead of answers avoided the learning loss entirely. And a 30-month study of 26,811 Chinese secondary students found that students who kept putting in real effort while using AI showed minimal learning loss.

Some caution is warranted. StudentBench is a preprint, funded by Handshake AI, and it measured learning right after one hour, not months later. Its participants were paid adults, not K-12 students. It also had no group that simply did practice problems alone, so we can't fully separate the tutor's value from the practice itself. Still, that last gap may be the point: the practice is where the learning happens.

What this means on Monday morning

The lesson for our classrooms isn't "use AI" or "avoid AI." It's that practice quality is what matters, and AI can either multiply it or replace it. A few ways to tip it toward the former:

• Judge a tool by who does the work. Before adopting one, try it as a student would. Does it ask you to attempt the problem first, or does it solve it for you? If the answer appears before the effort, learning will suffer.
• Start from real mistakes. The StudentBench tutors began with each student's actual errors. We can do the same: feed an AI tool a quiz's error patterns and ask for targeted practice sets, then check them for accuracy before students see them.
• Measure without the scaffold. Homework done with AI can look great while real skill stalls. Build in regular, unassisted checks, like an exit ticket or a short in-class problem, so we see what students can do on their own.

The rep is still the point

For most of history, one-on-one practice with immediate feedback was a luxury. This research suggests it could become far more common, but only if we insist that the student stays the one doing the work. The technology has changed. The way people learn hasn't. So the question to ask of any AI tool in our classrooms is simple: how many more good reps does it give my students?

This is part of Teaching in the Age of AI*, a weekly digest of research and ideas for educators navigating AI in the classroom. Subscribe, follow, or share to get each week's post.*

‍