AI literacy can only be seen over time
Whether a course or programme has developed AI literacy can’t be settled by an end-of-course assessment. It shows in whether participants go on engaging thoughtfully, adapt their practice as AI changes, and develop increasingly sophisticated judgement, and that takes months to observe.
What a course can claim about AI literacy
One-sentence definition: The limits on what any course can claim about AI literacy come from what literacy is: AI literacy develops through practice, grows in stages and differs by context, and can only be partly assessed when the course ends.
None of this reduces the value of structured learning. It clarifies what a module outcome such as “students will be AI literate” can honestly promise, and what a programme team would need to look for afterwards to know whether it had been met.
Literacy develops through practice
Learning resources provide frameworks, techniques, and conceptual foundations, but literacy develops through sustained practice, in the same way that taste and judgement develop through practice, reflection, and accumulated experience. A participant who reads the lessons and never applies them won’t be AI literate, just as reading about information literacy doesn’t make someone information literate.
Good course design anticipates this. Its activities require participants to work with AI on real tasks, reflect on what happened, and refine their approach, so that they’re using AI throughout the course as well as learning about it.
Literacy is developmental and contextual
AI literacy isn’t a state someone is either in or out of. People start where they are and work with AI in increasingly sophisticated ways, from using it as a substitute for tasks they already do towards changing how they work; developing AI literacy describes the stages and what moves people between them. Finishing a course marks a point on that trajectory.
It also varies by domain. What counts as meaningful engagement differs across research, teaching, and administration, so someone can be AI literate in one (using AI to draft and revise their writing, say) while still developing in another (using it to design a case-based seminar for occupational therapy students).
A course covers its own context
A well-designed course delivers AI literacy as the literacy frameworks define it, within its intended context. It won’t cover every aspect: training models, contributing to AI development, and building AI systems are usually out of scope, and that’s a reasonable design choice. Information literacy for academics differs from information literacy for journalists or data scientists, and AI literacy for scholarly work differs from AI literacy for other fields in the same way. A course for academics delivers the capability to recognise, evaluate, use, create with, reflect on, and make judgements about AI in scholarly practice, and the frameworks it teaches carry over to new contexts as they’re needed.
Assessing AI literacy
AI literacy, like all literacies, is easier to recognise than to measure. You can assess whether someone knows how to write a structured prompt, which is functional application; whether they’ve developed good taste about when and how to engage, which is contextual judgement, only shows in their practice over time.
Courses can get closer by building in reflection, documentation of learning, and tasks that grow in complexity, so that participants demonstrate developing literacy and not only acquired knowledge. The test that matters comes later: six months after the course, are participants still engaging with AI thoughtfully, adapting their practice as capabilities change, and applying the frameworks without being prompted to? Answering that means observing practice longitudinally, the way programmatic assessment builds up evidence over a programme instead of relying on a single end point.