AI-driven adaptive learning: the cheat sheet
One page on how adaptive learning software works: learner models (BKT, PFA, DKT, Elo, knowledge spaces), spacing schedulers, difficulty targets, and what to ask a vendor.
Every adaptive learning product does three things: it keeps a model of what the learner knows, it schedules when things come back, and it picks a difficulty to aim at. This sheet lays out the standard options for each, with what data they need, where they break, and the numbers from the papers that compared them. Peabody (peabody.kids), a free AI-driven adaptive learning platform for kids, is the worked example at the end: AI writes the stories; code decides the rest. It is free through the end of 2026.
How to use this: print it, keep the tables, skip the prose. Each table has a "where it breaks" column because that is the column vendors leave out. Sources are named in the rows and linked once; the caveats are the ones the papers themselves give.
Table 1: learner models (what the software believes the child knows)
| Model | How it works | Data needed | Good at | Where it breaks |
|---|---|---|---|---|
| Bayesian Knowledge Tracing (BKT), Corbett and Anderson 1995 | One hidden state per skill, known or not. Four parameters: P(L0) prior of knowing, P(T) chance of learning at each try, P(G) guess, P(S) slip. Each answer updates P(L) by Bayes' rule, then adds a learning step: P(L) + (1 − P(L)) · T. Mastery is declared at P(L) ≥ 0.95 by convention. | Hundreds of answer sequences per skill to fit four numbers per skill | A probability a teacher can read; the standard in cognitive tutors for thirty years | Parameters are not identifiable (several sets fit the same data equally well, van de Sande 2013), so fitted numbers cannot be read literally. Original bounds were G < 0.3 and S < 0.1; three-option multiple choice gives G ≥ 0.33 by blind guessing, so one right answer is weak evidence. One skill per item. |
| Performance Factors Analysis (PFA), Pavlik, Cen and Koedinger 2009 | Logistic regression: logit of P(correct) = skill easiness + γ · prior successes + ρ · prior failures, summed over every skill an item needs | About 100 or more learners per item; a map of items to skills (a Q-matrix) | Items that need several skills at once; about equal to BKT at prediction across many comparisons | No slip parameter, so it assumes accuracy heads to 100% with practice. Pelánek 2021 shows a better-fitting PFA can give worse predictions when the skill map is wrong. |
| Deep Knowledge Tracing (DKT), Piech et al. 2015, and attention successors (SAKT, AKT) | A recurrent or attention network reads the whole sequence of (skill, right or wrong) and predicts the next answer; no explicit skill state | Very large: Gervet et al. 2020 found DKT leads only above roughly one million interactions | Big datasets with many learners per item; finds structure nobody wrote down | Overfits on small or moderate data; opaque; the gain is small even where it wins (0.01 to 0.06 AUC in Gervet 2020). Khajah et al. 2016 found a properly fitted BKT with forgetting and ability matches DKT on average. The pyKT benchmark (Liu et al. 2022) found the same model's reported AUC on the same dataset ranged from 0.709 to 0.86 across papers because protocols differed. |
| Elo and Glicko-style ratings, Klinkenberg et al. 2011 (Math Garden), Pelánek 2016 | A rating θ per child and a difficulty d per item; P(correct) = 1 / (1 + e^(−(θ − d))). After each answer θ moves by K · (correct − P) and d moves the opposite way. Pelánek's uncertainty function shrinks K with the number of answers n: U(n) = a / (1 + b·n), fitted a = 1, b = 0.05. For multiple choice with k options: P = 1/k + (1 − 1/k) / (1 + e^(−(θ − d))). | Works from the first answer; about three hyperparameters; items and children rated online | Sparse data, new items, new children; handles guessing with the k-option form; recovers item difficulties about as well as a full Rasch fit, live | One number per child per skill (the multivariate version propagates updates to correlated skills but needs correlations from data). When skill is actually changing with practice the symmetric update is wrong; Pelánek et al. 2017 use asymmetric gains. |
| Knowledge Space Theory (ALEKS), Doignon and Falmagne 1985 | A knowledge state is the set of problems a learner can solve; the feasible states form a space, and each assessment question roughly halves the candidates, so about log2(number of states) questions place a learner. The "outer fringe" of a state is what the learner is ready for next. | A hand-built map of which problems presuppose which; ALEKS caps assessments at about 30 questions | A formal version of the zone of proximal development; placement in few questions | The map is expensive to build and the implementation uses a restricted class of spaces. No independent ALEKS effect size was located; a RAND trial found usage too low to show an effect. |
Primary sources: Corbett and Anderson 1995 (equations at https://en.wikipedia.org/wiki/Bayesian_knowledge_tracing); Gervet et al. 2020, JEDM (https://jedm.educationaldatamining.org/index.php/JEDM/article/download/451/123); Khajah, Lindsey and Mozer 2016, EDM (https://arxiv.org/abs/1604.02416); Pelánek 2016, Computers and Education (https://www.fi.muni.cz/~xpelanek/publications/CAE-elo.pdf); Liu et al. 2022, pyKT (https://arxiv.org/abs/2206.11460); Falmagne et al. 2013 on knowledge spaces (https://arxiv.org/abs/1511.06757).
Ranked by data appetite, from Gervet 2020, Pelánek 2016 and pyKT 2022: Elo with an uncertainty function (first answer onward), then BKT per skill, then PFA, then DKT or AKT (about a million interactions for a gain of 0.06 AUC or less). Language-model embeddings of question text help cold start for brand-new items (DKT falls to 0.5 AUC on items it has never seen) but zero-shot prompting was "not successful" at tracking knowledge (Neshaei et al. 2024).
Table 2: spacing schedulers (when things come back)
| Scheduler | Mechanism | Parameters | Reported result | Caveat |
|---|---|---|---|---|
| Leitner (1972) | Boxes. A right answer moves a card up one box; a miss sends it back to box 1. Higher boxes are reviewed less often. | The number of boxes and each box's gap | The baseline every later system is measured against | Fixed gaps; every child and card gets the same ladder |
| SM-2, Woźniak 1987 | First gap 1 day, second 6 days, then each gap is the previous one times an easiness factor EF. After each review EF changes by 0.1 − (5 − q)(0.08 + (5 − q) · 0.02) where q is self-rated quality 0 to 5; EF starts at 2.5 and never drops below 1.3; q below 3 resets the card to day 1. | EF per card, q per review | The basis of most flashcard apps for three decades | Depends on the learner grading themselves; never predicted a recall probability |
| Half-life regression, Settles and Meeder 2016 (Duolingo) | Recall probability p = 2^(−Δ/h) after Δ days; the half-life h = 2^(Θ · x) is learned from features x: counts of right and wrong, total exposures, item tags | A weight per feature, fitted on 13 million traces | 45% or more error reduction against Leitner and Pimsleur at predicting recall | Company data, company authors; a prediction result, not a learning result |
| FSRS, open source, an Anki option | Difficulty D from 1 to 10, Stability S (days until recall falls to 90%), Retrievability R. R(t, S) = (1 + FACTOR · t/S)^DECAY with DECAY = −0.5 and FACTOR = 19/81 in versions 4.5 and 5. Next gap = S/FACTOR · (R_desired^(1/DECAY) − 1), usually with R_desired = 0.9. | 17 (v4), 19 (v5) or 21 (v6) learned weights, flagged as verify in primary | On the authors' benchmark (August 2025): log loss 0.346 for FSRS-6 against 0.722 for SM-2; FSRS-6 beats SM-2 on 99.5 to 99.6% of collections; default parameters reach 0.366 | The benchmark is built by FSRS's own authors; SM-2 never predicted probabilities, so formulas were bolted on to score it; the metric is prediction, not learning; "20 to 30% fewer reviews" is from simulation, not a trial |
| Peabody's item ladder | A known item (level 3 or above) comes back after 1, then 3, then 7, then a 21-day gap, growing with each separate day it was answered quickly and right; a miss resets it | Four gaps, the same for every child | None claimed; readable off a table and fenced by a test that the gaps strictly increase | Not personalised; the gain from a per-child model is real for prediction and unshown for learning |
Sources: SM-2 (https://www-v1.supermemo.com/archives1990-2015/english/ol/sm2); Settles and Meeder 2016, ACL (https://aclanthology.org/P16-1174); FSRS algorithm wiki (https://github.com/open-spaced-repetition/fsrs4anki/wiki/The-Algorithm) and benchmark (https://github.com/ankitects/fsrs-benchmark).
What the spacing studies agree on: the best gap grows with how long you want to remember but shrinks as a share of it, about 20 to 40% of a one-week horizon and 5 to 10% of a one-year horizon, and too-short gaps hurt more than too-long (Cepeda, Vul, Rohrer, Wixted and Pashler 2008, Psychological Science, N = 1,354, https://pubmed.ncbi.nlm.nih.gov/19076480/). Delaying the first retrieval is the active ingredient, not the expanding shape (Karpicke and Roediger 2007, JEP:LMC, https://www.gwern.net/doc/psychology/spaced-repetition/2007-karpicke.pdf). In children's maths, spacing gives g = 0.28 across 27 studies while retrieval versus restudy has a confidence interval that crosses zero (Murray, Horner and Göbel 2025, Educational Psychology Review, https://link.springer.com/article/10.1007/s10648-025-10035-1).
Table 3: difficulty targets (how hard to aim)
| Target | Source | What it says | Scope caveat |
|---|---|---|---|
| The 85% rule | Wilson, Shenhav, Straccia and Cohen 2019, Nature Communications (https://pmc.ncbi.nlm.nih.gov/articles/PMC6831579) | For gradient-descent-like learners on binary classification, learning is fastest at an error rate of about 15.87% | The authors' own caveat: simple two-choice tasks with a clear right and wrong; not a prescription for classrooms |
| Math Garden's 0.75 | Klinkenberg et al. 2011; Hofman et al. 2018, Journal of Intelligence (https://www.mdpi.com/2079-3200/6/1/14/html) | Items are chosen so the child has about a 75% chance of being right, not the 50% that gives the most measurement information, because failing half the time is unacceptable in practice software | A design choice justified by engagement, then validated for reliable estimates |
| Zone of proximal development | Vygotsky 1978 | What a child can do with help but not yet alone | Formalised by ALEKS's outer fringe and by selecting items at about P(correct) 0.75 to 0.85 |
| Three-option correction | Arithmetic on the 85% rule | A raw 85% on three-box items is a true knowledge probability of about 0.78 once guessing is removed (0.85 = 1/3 + 2/3 · p) | So 80 to 85% raw with three boxes lands where Math Garden's 0.75 does |
| Wheel-spinning | Beck and Gong 2013, AIED | A student who does not get 3 in a row within the first 10 tries is likely stuck; common, not rare | Design rule: change representation, drop a level or hand off; never serve an eleventh identical item |
Table 4: what to ask a vendor
| Question | Why it matters | A good answer sounds like |
|---|---|---|
| Which learner model, and what decides the level: a model, rules, or a language model's opinion? | A language model guessing a level is not a model of the child | "Rules" or "Elo" or "BKT per skill", with the parameters stated |
| How do you handle guessing with multiple choice? | Three options means a third of blind guesses are right; plain BKT assumes G < 0.3 | "The k-option form", or a pass bar that needs many rights across a skill |
| What is the review schedule, and can I read it? | A schedule you can read can be checked; a fitted one needs a benchmark | A table of gaps, or a named model with its benchmark and caveat |
| What accuracy do you aim for during practice? | Below about 75% raw, children stop; above 95%, little is new | A number between 75 and 85% for new material |
| Is a question ever asked again after the child has it? | Most practice software repeats known items forever | A retire rule with a constant |
| Does the model use response time, and is a clock shown? | Pelánek 2016: a speed score is only fair when the deadline is visible | "Speed is measured, never demanded on new items" |
| What happens after ten misses on one skill? | Wheel-spinning is common | A change of tack, not an eleventh item |
| What is the independent effect size, on whose scale? | Vendor "used as recommended" subgroups are selection, not effect | A number on Kraft's 2020 scale with the trial named; or an honest "not yet tested" |
How Peabody does it
Peabody's practice brain is a few hundred lines of ordinary code; the full rulebook is in how Peabody picks the next question. The rows above map to it like this.
| Row | Peabody today |
|---|---|
| Learner model | No fitted model. Each item sits at a level 0 to 5 per child: 0 unseen, 1 missed last time, 2 right but hinted or with only two options, 3 right first try, 4 quick and right with three options, 5 quick and right on two different days. "Quick" is a fixed threshold per strand (3.0 s for adding, 2.5 s for times tables, 5 s for word meanings). |
| Why not plain BKT | Every answer is three big tap boxes, so a blind guess is right a third of the time, outside Corbett's G < 0.3. Peabody treats one right answer as weak evidence by design: level 5 needs quick rights on two separate days, and a rung passes only when 80% of what was seen is solid. A single question retires after one right, which a guess can earn; the skill comes back through fresh questions, so a lucky tap moves a child on from a question, not up the ladder. If a model ever replaces the rules it would be the k-option Elo form, which is built for exactly this. |
| Skills | A mastery ladder per subject: math climbs count, ten, add, sub, number sense, mul, div; reading climbs sound, rhyme, phon, sight, spell, vocab. A rung is passed when enough has been seen, 80% of it is solid, and few items are still shaky; or by test-out at 12 or more questions with 88% right. A grade only sets the first rung. |
| Spacing | Fixed ladder: 1, 3, 7, then a 21-day gap, reset by a miss. Story practice words use a separate Leitner-style ladder: three right cold reads on three different days, then keeper checks at 7, 30 and 90 days. |
| Difficulty target | A deck is about 30 questions and about 85% known items, due reviews first, then learning items, then at most two new items a deck and four a day per domain. Frustration guard: two misses in the last five and no new items this round; three and the game is told to ease. |
| Never twice | A question answered right first try is never asked again (the constant is 1); a miss is asked once more, later in the round, and is then done either way. Facts retire question by question; word items have one question per key. |
| Stretch | A child who is cruising gets the next six items in order, untaught; a thin deck climbs until a miss, which works as placement. A struggling child gets no stretch. |
| Measurement | A probe card about once every 40 answers moves no level. Peabody doesn't measure Lexile and cannot hear a child read. |
| Invariants | A test suite fences every game: must ask the brain, three boxes, never-twice constant = 2, 80% pass share, increasing review intervals, ladders in grade order, simulated children never get an empty deck. |
The split is the same everywhere: AI writes the stories; code decides the rest. Peabody has no published efficacy study, so none of the effect sizes above are claims about it.
Limits of this sheet
- Prediction is not learning. Every AUC and log-loss number above measures how well a model predicts the next answer, not whether a child learns more under it.
- Most knowledge-tracing comparisons are on older students doing maths homework (ASSISTments, Statics). The developmental trajectory of spacing and retrieval in five- to twelve-year-olds is poorly mapped (Fazio and Marsh 2019).
- Several numbers in the source report are flagged as needing a check in the primary paper: the BKT bound relaxation to 0.5, PFA's exact notation, and FSRS's parameter counts.
- The pyKT finding applies to everything here: a reported AUC without its evaluation protocol is not comparable to any other.
You can watch the engine from the outside with a free grown-up account, or just start free on any browser and see which questions come back when.
Questions parents ask
Is any of this AI?
In Peabody, no. The learner model, the schedule and the difficulty target are ordinary code with the numbers in this sheet. A language model writes the stories and screens them for safety. In other products the answer varies, which is why the first vendor question is "what decides the level".
Which model is best?
It depends on data. Gervet et al. 2020 found logistic models with the right features win on small and moderate datasets, and deep models win only above about a million interactions, by 0.06 AUC or less. For a product with few learners per item, an Elo-style tracker with an uncertainty function is the practical choice, and readable rules are a defensible one.
Why does three-option multiple choice matter so much?
Because a blind guess is right a third of the time. A model that assumes a low guess rate will believe a child knows something after one lucky tap. The fixes are either to put 1/k into the model (the k-option Elo form) or to make a skill count only after many rights across many questions, which is what Peabody does: one right retires a single question, but passing a rung needs 80% of what was seen to be solid.
Does a fitted spacing model beat a fixed ladder?
At predicting recall, yes: FSRS and half-life regression both report large gains over Leitner and SM-2, on their makers' own benchmarks. At making a child learn more, nobody has shown it in a trial. Cepeda 2008 and Karpicke and Roediger 2007 say the big wins are spacing at all and delaying the first retrieval, which a fixed expanding ladder already does.
What should I look for in a vendor's evidence?
An independent trial, the effect size on Kraft's 2020 scale (0.20 is large for a scaled intervention on a standardised test), and the whole enrolled sample rather than a "used as recommended" slice. Peabody has none yet and says so.
Checked against how Peabody works on 16 October 2026. Written with help from AI. Peabody is a reading app, not a reading assessment: if you're worried about your child's reading, talk to their teacher.