According to Ethan Mollick on X, Wharton-affiliated researchers including Hamsa Bastani found that giving high school students unrestricted ChatGPT access improved practice accuracy but led to worse exam performance without AI, while AI prompted to act as a tutor improved learning in controlled settings (as reported by Bastani et al., SSRN working paper and PNAS preprint references). According to Anand Sanwal citing the study, students using basic ChatGPT scored 17% lower on no-AI exams than those practicing without technology, indicating overreliance on answer-giving rather than reasoning. According to the SSRN paper by Bastani et al., a separate randomized controlled trial showed that structured tutor prompts increased learning outcomes, suggesting that guardrailed, step-by-step tutoring mitigates shortcutting. As reported by the authors, business opportunities include AI tutoring systems with scaffolded prompts, process feedback, and exam-transfer alignment for K–12 math, as well as district-level deployments focusing on formative assessment and metacognitive coaching.

According to Ethan Mollick on X, the authors of an earlier paper warning that students harm learning by copying AI answers now report a new randomized controlled trial showing that well-prompted AI tutors improve learning outcomes, as documented in their SSRN preprint. According to the SSRN paper, structured prompting and step-by-step guidance led to measurable gains versus control groups, indicating that guardrailed AI tutoring can enhance mastery rather than shortcut learning. As reported by the SSRN preprint, the study’s RCT design isolates the causal effect of prompt-engineered AI tutors, highlighting implementation details—such as scaffolded hints and verification steps—that schools and edtech firms can operationalize to reduce answer-copying and increase concept retention. According to the SSRN source, these findings open business opportunities for learning platforms to productize prompt frameworks, formative feedback loops, and analytics that track tutor adherence and student progress.

According to Ethan Mollick, a new randomized controlled trial shows that dedicated AI tutors measurably improve student learning outcomes, contrasting prior evidence that answer-only AI use undermines learning, as reported via the SSRN preprint (authors’ RCT) and Mollick’s commentary. According to SSRN, the study isolates structured AI tutoring from generic answer generation, indicating gains in assessment performance when students receive guided explanations and step-by-step feedback rather than direct solutions. As reported by the SSRN preprint, the RCT design strengthens causal claims on efficacy, suggesting institutions can deploy AI tutors to raise mastery while curbing shortcut behaviors.

According to @emollick citing @hamsabastani, a 5-month randomized field experiment in Taipei high schools found that combining an LLM tutor with reinforcement learning for adaptive problem sequencing improved final exam performance by 0.15 standard deviations across 770 Python students, with larger gains for beginners. According to Hamsa Bastani’s thread, all students used the same AI tutor and course materials; only the sequencing differed (adaptive vs fixed), isolating the effect of the reinforcement learning policy on learning outcomes. As reported by the study author, the mechanism appears to be stronger engagement and more productive AI use, inferred from student–chatbot interaction signals and solution attempts. According to the author’s summary, the system personalizes the next problem using interaction data, suggesting a scalable path for edtech providers to enhance outcomes without changing core content. For businesses, according to the thread, this points to opportunities to layer RL-based curriculum sequencing atop existing LLM tutors to drive measurable, test-verified learning gains and target novice learners for outsized ROI.