Michael Rowe

Epistemic accountability

note5 min read

Education catches errors because sources can be held to account.

Epistemic accountability describes the structural features that allow us to check whether a knowledge claim should be trusted — not just whether it is accurate now, but whether it can be challenged, verified, and held to account over time. Generative AI lacks these mechanisms, which changes the assumptions education can make about how errors get caught and corrected.

Epistemic accountability

One-sentence definition: Epistemic1 accountability is the set of structural features of an information source that enable its knowledge claims to be challenged, traced back to evidence, and evaluated against the source’s track record.

Traditional information sources carry three accountability mechanisms that learners rely on, often without noticing:

Interrogability. A student can ask a teacher to justify a claim, and the teacher must defend or retract it. This creates a real-time feedback loop in which the source is accountable to challenge. Generative AI largely inverts this: large language models tend towards sycophancy, agreeing with incorrect user statements rather than defending accurate ones, at the moment correction matters most (Sharma et al., 2023; Chen et al., 2025).

Traceability. Textbook claims can be traced to their evidential origins and compared against the broader literature. A lecturer’s claims can be evaluated against their expertise and publication record. LLM-generated information cannot be traced in the same way: even when AI provides citations, studies find high rates of fabricated or inaccurate references (Buchanan et al., 2024; Dahl et al., 2024).

Reputational cost. Human sources — teachers, textbook authors, clinicians — bear professional consequences for persistent errors. Each AI interaction is independent; there’s no accumulated track record a learner can draw on to calibrate trust, and no cost to the system for being wrong.

These three mechanisms are how educational environments normally catch and correct errors over time. When the information source actively undermines all three, the conventional assumption that education tolerates imperfection needs re-examination, which is the qualification the AI tutor accuracy essay develops.

Where undetected errors have consequences

The gap matters most in high-stakes learning contexts — health professions education, legal training, any domain where undetected errors have real-world consequences — because that’s where students most need to check what they’re told and are least equipped to spot an answer that’s plausible but wrong.

A second-year pharmacy student checks a paediatric dose with a chatbot, doubts the answer, and suggests a different figure. If the model agrees with the student’s wrong figure, nothing in the exchange marks the error. There’s no tutor who’ll pick it up next week, no reference behind the answer to check, and no record that this source was wrong the last time.

At the systems level, the concept maps to concerns being raised independently about AI at scale. Marchal et al. (2026) frame falsifiability and auditable reasoning chains as foundational requirements for trustworthy epistemic AI agents, which amounts to designing accountability into AI systems from the ground up. Their three trustworthiness properties (demonstrable competence, falsifiability, and epistemic virtues) map onto interrogability, traceability, and reputational cost, arrived at independently from the systems-design side rather than the learner’s.

Whether design can close the gap

Whether the accountability gap can be closed by design — through AI systems that maintain source trails, flag uncertainty, and resist sycophantic responses — is an open question. Some individual systems will become more accountable. What matters for learners is whether the mechanisms become reliable enough, and are adopted consistently enough, to restore the error correction they currently depend on.

Sources

  • Buchanan, J., Hill, S., & Shapoval, O. (2024). ChatGPT hallucinates non-existent citations: Evidence from economics. The American Economist, 69(1), 80–87. https://doi.org/10.1177/05694345231218454
  • Chen, S., Gao, M., Sasse, K., Hartvigsen, T., Anthony, B., Fan, L., Aerts, H., Gallifant, J., & Bitterman, D. S. (2025). When helpfulness backfires: LLMs and the risk of false medical information due to sycophantic behavior. npj Digital Medicine, 8, 605. https://doi.org/10.1038/s41746-025-02008-z
  • Dahl, M., Magesh, V., Suzgun, M., & Ho, D. E. (2024). Large legal fictions: Profiling legal hallucinations in large language models. Journal of Legal Analysis, 16(1), 64–93. https://doi.org/10.1093/jla/laae003
  • Marchal, N., Chan, S., Franklin, M., Revel, M., Keeling, G., Fischli, R., Chandra, B., & Gabriel, I. (2026). Architecting trust in artificial epistemic agents (arXiv:2603.02960). arXiv. https://arxiv.org/abs/2603.02960
  • Rowe, M. (2026). AI tutor accuracy in health professions education: The accuracy-engagement paradox. University of Lincoln.
  • Sharma, M., Tong, M., Korbak, T., Duvenaud, D., Askell, A., Bowman, S. R., Cheng, N., Durmus, E., Hatfield-Dodds, Z., Johnston, S. R., Kravec, S., Maxwell, T., McCandlish, S., Ndousse, K., Rausch, O., Schiefer, N., Yan, D., Zhang, M., & Perez, E. (2023). Towards understanding sycophancy in language models (arXiv:2310.13548). arXiv. https://arxiv.org/abs/2310.13548

Notes

  • Developed as part of the qualification to the accessibility paradox in the AI tutor accuracy essay — specifically the argument that AI-sourced errors resist correction differently from traditionally sourced errors

Footnotes

  1. Epistemic means to do with knowledge: how we come to know something, and how we judge whether a claim is justified. A clinical supervisor who asks a student “how do you know that?” is asking an epistemic question. Wikipedia: Epistemology ↩

Continue reading

  • AI literacy

    What is AI literacy? Six connected capabilities, from recognising AI systems to judging when they serve your goals, that have to develop together.

  • Large language models

    A plain explanation of large language models: what they are, how they are trained, where their limits come from, and what their ability to produce fluent text means for assessment in health professions education.

  • Similarities between AI and human thinking: What if we're the language models?

    Discussion of AI and human cognition usually dwells on the differences. Turn the question around and the terminology of language models describes human thinking surprisingly well: context windows, biased training data, tokenisation, temperature, hallucination, system prompts, and pattern matching all have human counterparts, some of which health professions education already has names for. The similarities say something about our own cognitive architecture, and the strength of our resistance to them says something about what our professional identities depend on.