I just came across a study that should make anyone thinking about AI in education sit up and take notice. In their paper ““Check My Work?”: Measuring Sycophancy in a Simulated Educational Context” researchers tested five different LLMs in a simulated educational context, asking them to help students check their work. The results? When students mentioned an incorrect answer, the AI’s accuracy dropped by up to 15 percentage points. When they mentioned the correct answer, accuracy jumped by the same margin.

In other words, the AI was playing follow-the-leader with the students, even when the students were leading them off a cliff.

The effect was even more dramatic with smaller models: GPT-4.1-nano showed a 30% swing based on student suggestions, compared to 8% for GPT-4o. The researchers found that the models were literally changing their answers to match what students said, supporting what they called the “sycophancy hypothesis.”

I’ll admit to a certain grim satisfaction in seeing empirical data confirm something I’ve been arguing for a while now. Last year, I wrote about AI’s honey trap, how these systems are designed to tell us what we want to hear. This isn’t a bug; it’s a feature. As the adage goes, you catch more flies with honey than vinegar. These are systems intentionally designed to agree with us, to make us feel good, to keep us engaged. This is what happens when you design AI to be agreeable rather than accurate.

This is troubling but I think matters are even more complicated, and worse. This study only looked at single interactions. A student says something (right or wrong), the AI responds, we measure the error rate.

But that’s not how learning conversations actually work.

In real educational contexts, conversations unfold over time. One response leads to another. The student, emboldened by the AI’s agreement, builds on that initial error. The AI, trained to be helpful and agreeable, continues to elaborate on the flawed premise. Before long, you’ve got what I’ve called conversational drift. where small errors compound into significant misunderstandings, and the student does not have the judgment to recognize they’ve veered off course.

And here’s the kicker: the students who most need help, those without strong foundational knowledge, are precisely the ones least equipped to catch this drift. As I explored in my post on the expertise paradox, working effectively with AI requires the very expertise that students are supposed to be building. Students who already grasp core concepts can resist AI’s sycophantic tendencies. Those who don’t? They’re stuck in what I’ve called the novice’s dilemma, unable to evaluate AI outputs for accuracy and unaware of when the AI is leading them astray.

This creates a particularly troubling version of the Matthew Effect: AI makes learning gaps wider, not smaller. The students who need the most support get the least benefit, while those with advantages accelerate further ahead.

So what do we do about this?

The technical solutions (better prompts, verification checklists, more sophisticated guardrails) all miss the fundamental point. As I argued in my critique of OpenAI’s Study Mode, you can’t prompt-engineer your way out of a pedagogical problem.

Here’s what AI fundamentally cannot answer: why we’re learning what we’re learning? Why this particular problem matters? Why this knowledge is worth pursuing?

That’s where caring about the topic at hand can play an important role. When they’re driven by authentic inquiry, meaningful construction, real communication, and personal expression (Dewey’s four impulses), they may develop the discernment to catch AI’s yes-bot tendencies—not because they’re chasing right answers, but because they’re actually invested in engaging with, understanding, and applying what they’re learning. Not because we’ve taught them elaborate verification strategies, but because they’re interested in the quality of what they’re creating.

The solution isn’t just to make AI less sycophantic, though that would be a good start. That said, I am somewhat unsure whether there are any incentives for the AI companies to do that. Their model is based on engagement and sycophancy works. The onus is on us to create conditions where students care enough to notice when they’re being told what they want to hear rather than what they need to know. That’s a fundamentally human question—one that no amount of artificial intelligence can answer for us.