Literacy is longitudinal
The true test of whether a course or programme develops AI literacy cannot be answered through end-of-course assessment. It requires observing whether participants continue to engage thoughtfully, adapt their practice as AI evolves, and develop increasingly sophisticated judgement—which is precisely what literacy means.
A process that claims to develop AI literacy needs important qualifications. These caveats do not diminish the value of structured learning but clarify what literacy actually requires.
1. Literacy requires practice, not just instruction
Learning resources provide frameworks, techniques, and conceptual foundations, but literacy develops through sustained practice. Taste and judgement “develop through practice, reflection, and accumulated experience.” A participant who passively reads lessons but does not apply them will not be AI literate, just as reading about information literacy does not make someone information literate.
Effective course design anticipates this. Activities throughout should require actual AI engagement, reflection on outcomes, and iterative refinement. The progression from substitution through adaptation to transformation assumes participants are actively working with AI throughout, not just learning about it.
The true test of whether a course delivers AI literacy is longitudinal: do participants continue to engage with AI thoughtfully six months after completion? Do they adapt their practice as AI capabilities evolve? Do they develop increasingly sophisticated judgement about meaningful engagement?
These questions cannot be answered through end-of-course assessment. They require observing whether participants have internalised frameworks and continue applying them independently—which is precisely what literacy means.
2. AI literacy is developmental and contextual
AI literacy is not binary—literate or illiterate—but developmental. Participants begin where they are and progress through increasing sophistication. The substitution-adaptation-transformation framework itself acknowledges stages of development.
Additionally, AI literacy is domain-specific. What constitutes meaningful AI engagement differs across research, teaching, and administration. Someone can be AI literate in one domain (e.g., using AI for writing) while still developing literacy in another (e.g., using AI for teaching).
This means that completing a course marks a point on a developmental trajectory, not an endpoint. Continued growth requires ongoing practice, reflection, and adaptation as both AI capabilities and professional contexts evolve.
3. Courses deliver foundational AI literacy for specific contexts
A well-designed course delivers AI literacy as defined by literacy frameworks within its intended context. It does not address every possible aspect of AI literacy—for example, it may not cover training AI models, contributing to AI development, or technical implementation of AI systems.
This scoping is appropriate. Just as information literacy for academics differs from information literacy for journalists or data scientists, AI literacy for academic workflow differs from AI literacy for other domains. A course for academics delivers what academics need: the capability to recognise, evaluate, use, create with, reflect on, and make judgements about AI in their scholarly practice.
The limitation is not a flaw but a design choice. Foundational literacy in one context provides transferable frameworks that support developing literacy in new contexts as needed.
The challenge of assessment
AI literacy, like all literacies, is easier to recognise than to measure. You can assess whether someone knows how to write a structured prompt (functional application), but assessing whether they have developed good taste about meaningful engagement (contextual judgement) requires observing practice over time.
Courses can address this by emphasising reflection, documentation of learning, and progressive complexity. Activities create opportunities for participants to demonstrate developing literacy rather than just acquiring knowledge. But the ultimate measure of literacy is whether participants continue applying what they have learned independently, adapting their practice as contexts change.