The gold standard would be to make each student do an oral defense of their work. Hand-written exams are good too. Both of these approaches are time-consuming to mark, though.
Assessment design can require students to perform analysis of rare and obscure source material, so that there are no published materials for AI to draw on (which can make it struggle). Similarly, asking for a good bibliography that lists the sources used in the text and formats it correctly would show up the hallucinatory tendencies of AI, I think — though I haven’t tried the latest models.
It works great. That’s basically how it’s done in aviation, flight instruction is inherently 1 on 1, so I’d assign my students to do some reading and then it was pretty obvious if they’d done that the next flight based on who was asking questions. The ones that did the reading will come in asking me to clarify something, if they say “Yeah I read it” and I have to do the clarifying they didn’t read it. Oral and practical tests are the standard, show me you can fly the plane.
To make that work, most professional pilots have to be flight instructors at some point. It’s almost certainly the first flying job you’ll get after earning a commercial certificate, and thus spend at least some time doing that job. You’re not going to get that in most college classes, so where are you going to get the labor necessarily to give an oral exam to every student in every class?
Every exam should have some oral component. Doesn’t need to be the whole of one’s grade, but 10% sounds about right: significant enough to be impactful, but not so significant that people with severe anxiety can’t otherwise pass.
The gold standard would be to make each student do an oral defense of their work. Hand-written exams are good too. Both of these approaches are time-consuming to mark, though.
Assessment design can require students to perform analysis of rare and obscure source material, so that there are no published materials for AI to draw on (which can make it struggle). Similarly, asking for a good bibliography that lists the sources used in the text and formats it correctly would show up the hallucinatory tendencies of AI, I think — though I haven’t tried the latest models.
Doesn’t scale.
It works great. That’s basically how it’s done in aviation, flight instruction is inherently 1 on 1, so I’d assign my students to do some reading and then it was pretty obvious if they’d done that the next flight based on who was asking questions. The ones that did the reading will come in asking me to clarify something, if they say “Yeah I read it” and I have to do the clarifying they didn’t read it. Oral and practical tests are the standard, show me you can fly the plane.
To make that work, most professional pilots have to be flight instructors at some point. It’s almost certainly the first flying job you’ll get after earning a commercial certificate, and thus spend at least some time doing that job. You’re not going to get that in most college classes, so where are you going to get the labor necessarily to give an oral exam to every student in every class?
Agreed 100%.
Every exam should have some oral component. Doesn’t need to be the whole of one’s grade, but 10% sounds about right: significant enough to be impactful, but not so significant that people with severe anxiety can’t otherwise pass.