Text that is easy to read feels true
When information is easy to process, people judge it as more likely to be true, more familiar, and better than the same information presented less clearly. The ease belongs to the text and says nothing about whether the claim is right, but readers rarely separate the two. AI-generated text is about as easy to process as text gets, so it reads as credible whether or not it’s accurate.
What is processing fluency?
One-sentence definition: Processing fluency is the subjective ease with which a person perceives, reads, or understands a piece of information.
People don’t usually notice fluency as such. They experience it as a feeling that something makes sense, and they use that feeling as a cue when they’re judging something else. Alter and Oppenheimer (2009) reviewed the research and found the same pattern across many sources of ease, including clear print, simple words, familiar phrasing, and repetition, and across many kinds of judgement: information that’s easier to process is rated as more true, more likely, more familiar, and more pleasing, and people feel more confident about it. Reber et al. (2004) made the case that much of what people experience as beauty comes from the same source.
Reber and Schwarz (1999) showed the effect on judgements of truth directly. They presented factual statements printed in colours that were either easy or harder to read against the background, and people judged the easier-to-read statements true more often. The claims were identical, so the only thing that changed was how readily they could be read.
Most of the time the habit makes sense, because things we’ve met before are easier to process, and claims we’ve met often are more likely to be true, while an argument that’s easy to follow has often been well put together by someone who understood it. The cue fails when something other than truth makes information easy to take in, such as a false claim repeated often enough or a weak argument presented well.
Processing fluency and AI-generated text
Language models produce text that’s grammatical, well organised, and phrased in the most familiar way available, because predicting probable wording is what they’re trained to do. That’s close to a description of maximum processing fluency, and it holds whether the content is accurate or not: a hallucinated reference reads as smoothly as a real one. So the ordinary reading instinct gives a fluent wrong answer the same credibility as a fluent right one, which is the argument of AI fluency is noise to be filtered out.
Health professions education has relied on this instinct in assessment for a long time. An occupational therapy lecturer marking case studies has spent years reading clear writing as a sign of clear clinical reasoning, and with student work that inference was usually sound, because producing a coherent case study took the reasoning that went into it. A case study drafted with a language model can be just as coherent with much less reasoning behind it, and the marker’s sense that it’s strong is now partly a response to how easily it reads. Separating the two judgements has to be done on purpose: set the prose aside, state what the student has claimed about the client, and ask what in the case supports it.
Processing fluency and AI fluency
The phrase “AI fluency” is used for both of these, and they’re different things. Processing fluency is a property of text as a reader experiences it. AI fluency is a person’s ability to work with AI systems effectively, efficiently, ethically, and safely. They meet in discernment, the AI fluency competency of judging what an AI system produces, because judging AI output well means discounting how fluent it is.
Sources
- Alter, A. L., & Oppenheimer, D. M. (2009). Uniting the tribes of fluency to form a metacognitive nation. Personality and Social Psychology Review, 13(3), 219–235. https://doi.org/10.1177/1088868309341564
- Reber, R., & Schwarz, N. (1999). Effects of perceptual fluency on judgments of truth. Consciousness and Cognition, 8(3), 338–342. https://doi.org/10.1006/ccog.1999.0386
- Reber, R., Schwarz, N., & Winkielman, P. (2004). Processing fluency and aesthetic pleasure: Is beauty in the perceiver’s processing experience? Personality and Social Psychology Review, 8(4), 364–382. https://doi.org/10.1207/s15327957pspr0804_3