Skip to the main content
PeabodyMake a free story

What makes AI learning software work?

Eight questions about what the 2025 and 2026 studies found. One best answer each, and every answer tells you why, so you learn as you go.

Question 1 of 8

In a two-year trial in 18 middle schools, what did adding the Khanmigo AI tutor to Khan Academy practice add?

The best answer to each question, and why

  1. In a two-year trial in 18 middle schools, what did adding the Khanmigo AI tutor to Khan Academy practice add? About what the practice already gave without the AI. Right. The researchers found gains like those from Khan Academy practice with no AI. The practice did the work; the chat window added little.
  2. In that trial, how often did students open the AI tutor in practice sessions where they had made a mistake? In about one session in six. Yes, about 17%. Nearly everyone tried it once, but few came back. A tutor the child has to choose to open mostly goes unused.
  3. When does AI help seem to do the most good in practice? Right after a mistake, inside a structured practice system. Right. In a study of 6,000 middle schoolers the clearest benefit was on the next try after a wrong answer, and it worked best combined with mastery practice.
  4. Students practised maths with a plain chatbot that would give answers. What happened when it was taken away for the exam? They did up to 17% worse than students who never had it. Yes (Bastani et al., PNAS 2025). They leaned on it as a crutch. A version that gave hints, not answers, avoided most of the harm.
  5. What should a learning app use to decide what your child knows? A statistical model, ideally with a measure of how sure it is. Right. Ordinary statistical trackers still lead. The best ones keep an error bar, so one lucky guess doesn't move a child up.
  6. What's the best way to tell whether a child has learned something? They answer a new question on it, alone, days or weeks later. Yes. A delayed question, with no help and not one they've seen, is the closest thing to the real test. The best software checks skills this way.
  7. Your child gets four questions wrong in a row. What should a good app do next? Make the next one easier, or show it a different way. Right. Step down, switch to pictures or blocks, or show a worked example. Most answers should be wins.
  8. Which kind of AI help did best in the trials? A tutor built around one task, that already knows the answer and goes one step at a time. Yes. The Harvard physics tutor and others with real gains all had a fixed task, a checked solution and scaffolding the software enforced, often with a person nearby.

Read before you decide

More quizzes