Michael Rowe

Sycophancy

note4 min read

Agreement from an AI is not evidence that you're right

A language model tends to agree with you: it praises the plan you bring, confirms the idea you favour, and backs down when you push back, even when it had the right answer first. This comes from how models are trained, so it doesn’t go away on its own. You have to ask for the disagreement you need.

Sycophancy

One-sentence definition: Sycophancy is a language model’s tendency to give the response the user wants to hear over the one that is accurate.

Why AI agrees with you

Language models are refined using human feedback:1 people compare pairs of responses and pick the better one, and the model learns to produce more of what gets picked. People tend to prefer responses that agree with them.

Sharma et al. (2023) tested five widely used AI assistants and found the same pattern in all of them: they gave feedback on an argument that shifted with the opinion the user said they held, admitted to mistakes they hadn’t made when the user asked are you sure?, and repeated errors the user had introduced. When the researchers looked at the human preference data behind the training, responses that matched the user’s views were more likely to be preferred. Both the human raters and the preference models2 trained on their judgements sometimes chose a convincing, agreeable answer over a correct one. That’s why the same behaviour turns up across products: it’s what you get when a system is trained to be rated well by the person it’s talking to.

What it looks like in practice

It shows up in three ways: the model flatters (that’s an excellent question), agrees with a claim or plan because you presented it as yours, and caves when you disagree with a correct answer, apologising and changing it. The flattery is an irritation. The agreeing and the caving are the problem, because they happen when you’re relying on the model to check your thinking. A plan you ask the model to review will usually come back improved at the edges and endorsed in the middle.

Asking for criticism helps: argue against this plan, what would a sceptical colleague say?, or giving the output to a second model and asking what’s wrong with it. None of this removes the tendency, but it changes what the agreeable response is.

In clinical and educational settings

Chen et al. (2025) gave five widely used language models medical requests built on a false premise, such as asking them to explain why one drug is safer than another when the two are the same drug under different names. The models complied at rates up to 100%, writing persuasive misinformation instead of pointing out the error.

For a student, the risk sits where the learning should happen. A nursing student pastes a reflective essay about a difficult placement into a chatbot and asks how to improve it. The reply calls the reflection honest and insightful, suggests tightening one paragraph, and says nothing about the fact that the essay describes what happened without ever asking why it happened or what the student would do differently next time. A practice supervisor would press on exactly that, because the analysis is what reflective writing is for. The student comes away more confident in a reflection that hasn’t yet begun, with no way of knowing how little the praise was worth. That missing signal is part of the epistemic accountability gap between AI and the sources education has relied on.

The same applies to clinicians and educators: a model asked to review a treatment plan, an assessment brief, or a piece of patient information will find it easier to agree than to object.

Where hallucination is the model saying something false on its own, sycophancy is the model going along with something false because you said it. AI literacy needs to cover both.

Sources

Footnotes

  1. Reinforcement learning from human feedback — the training stage in which people rate a model’s answers and the model is adjusted towards the kind of answer that gets rated highly. It’s what turns a system that predicts text into an assistant that follows instructions. Wikipedia ↩

  2. Preference model — a second model trained on thousands of those human ratings to predict which of two answers a person would pick. It stands in for the human raters during training, so whatever the raters favoured, it favours too. Wikipedia ↩

Continue reading

  • Large language models

    A plain explanation of large language models: what they are, how they are trained, where their limits come from, and what their ability to produce fluent text means for assessment in health professions education.

  • Hallucination

    When a language model produces a fluent, confident statement that is false, why it happens, and why it is harder to catch than other AI errors.

  • Epistemic accountability

    The structural features of an information source that enable its knowledge claims to be challenged, traced back to evidence, and evaluated against the source's track record. Traditional sources carry it; generative AI largely does not.