My wife is a teacher, she has shown me vibed handed in assignments abd its incredibly obvious.
Right off the bat, if she gives an assignment to make, say, a slideshow on “Topic” and they talk about a examples A, B, and C in class, and the assignment goes off on tangents about topics F, G, and H instead, it’s an instant red flag.
This happens cuz the student just copy paste the assignment blurb into gpt, but gpt has no context for what was discussed in class… so it goes off the rails instantly.
Its also easy to include poison pills in the middle of an assignment if they copy paste it straight into gpt.
Also theres all the usual markers. Emoji, em dash, and the assignment having way higher verbosity than you know damn well the kid has the vocabulary for. Suddenly they’re speaking at a grade 7~8 levels higher than usual? Uh huh. .
From her and her teacher friends, Ive been told its extremely obvious to spot still. And its pretty trivial to setup the assignment to poison pill the AI.
Ah. Good old days in the 80’s when teacher didn’t even read what you wrote. Grade was given according who you were, did the teacher like you and what your previous grades were. No sudden inspirations to do better.
This happened to me. I was a pretty good kid; brainy too, but one history test kicked my ass. The teacher was the husband of my grandfather’s sister. A distant family cut me a break.
This phenomenon can continue into adulthood too.
However, after a certain point it’s not about the grades you make. It’s about the hands you shake.
Sure as hell ain’t my students; it’s been a steady decline since ChatGPT came out and I think I may have failed more students than ever over unfinished projects. You can’t GPT the semester long project, there’s a paper trail and data to collect can it becomes super clear who is AI brained now…
Edit: PS, grade inflation has been a thing for a few decades now, btw; the As aren’t the problem so much as the mush brain.
You are unfair, you don’t let them use ChatGPT during tests 😉
They’re not grading ChatGPT performance either
Isn’t that how it is suppose to work?
At the end of class you get a grade.
Education needs to change. Including punishment for using LLMs.
I dunno, they’re here to stay. Cat’s out of the box. Educators and education need to adapt. In person assessment is probably the ideal way to gauge progress and learning, but due to resources I don’t see it being practical.
they’re here to stay. Cat’s out of the box.
People keep saying this as though it’s true. The odds that this current era of free and ubiquitous access to these frontier LLMs lasts forever are pretty slim.
How do you figure? There are open source self host able solutions right now.
You can’t run anything like a frontier model on a self hosted solution. To get anywhere close you’d have to spend thousands of dollars on hardware which obviously isn’t free, or even a viable solution for the vast majority of people, let alone these students. And the quality of output you’d get from a model running on off the shelf consumer hardware like a MacBook is much more noticeably AI generated and trivial for AI detection tools to flag.
Already, very few middle schoolers have the tech savvy to self-host anything. If it’s not a tablet, they have trouble using it.
Add to that the possibility that the data center run on memory and processors could mean that local computing power will disappear, to be replaced with devices like Chromebooks that use corporate cloud services for everything.
Before you can punish for using LLMs, you need to be able to reliably detect the use of LLMs, including guarding against false positives.
Current AI checkers are woefully inadequate and prone to errors.
Before you can punish for using LLMs, you need to be able to reliably detect the use of LLMs, including guarding against false positives.
You can tell they’re using an LLM if they have a computer out during the pen-and-paper test.
Not US, but there’s a tendency of focusing more on the work during the semester than in the exam itself
LLMs are going to be a massive headache for me when they get older
How is that allowed?
Hell, back in my day, teachers were even very picky about what kind of calculator you could use. And if it was a graphing calculator, you had to show them yourself wiping the memory at the beginning of the test.
(Except for one algerbra teacher, who was really cool about it. He’d allow custom programs to stay on the calculator if you programmed it yourself. On the theory that if you can write a computer program that reliably solves these math problems, then you must have a very good understanding of how to solve these math problems. And, yes, I was one of the few kids who actually did that. Ah, writing my own custom software for the TI-83 on the TI-83, because that seemed easier than actually doing the math problems by hand … good times.)
We are allowing LLMs for all of our homeworks. As long as you can solve the problems in the indicated way with a reasonable answer.
In case you are not sure about the “indicated way”, there are practice questions with detailed step-by-step solutions for each hw problem that you just have to change the numbers/equations a bit and you’ll get points.
What we’ve noticed is that the year-after-year averages are significantly higher, especially this year. However, students are bringing in details that we explicitly didn’t go over in lecture and putting that on the homework (e.g. Delayed branching in Computer Architecture, because it’s a random quirk of MIPS that even assembly programmers don’t have to deal with). None of these details are ever mentioned in lecture or the practice homeworks (in a few cases, they are mentioned with the explicit wording “do not worry about this now”)
We can only assume people are copying the homework into LLMs and copying the results straight down. The latest exam had a question where students were asked to analyze a specific chunk of assembly code to deduce certain properties about it. Approximately 20-30% of the students didn’t know the FORMAT to answer it, despite it literally being item 1 on last week’s homework.
And when I say format, I don’t mean exactly “you must write these exact words or you lose points”. It’s literally just point out “line A and B have this property X because of attribute Y”. Just including ABXY as shown in the practice homework is enough. But apparently people are too lazy to read a 10 bullet point answer…
Then why issue homework at all?
Because the goal is to get people to learn/think about something. We don’t care what you use as long as you retain knowledge taught in the course. If what helps you learn is LLMs, then go for it.
Problem right now is there is a significant amount of people that are using these tools to do the thinking for them. And this is when Office Hours, Homework feedback, Email (I guarantee all students emails are responded to within 24hrs. Most are handled within 30 minutes) are all available and paid for (by tuition). I am even happy to schedule one-on-ones if privacy is a concern, but none of this is being utilized.
Then those people will go and fucking vote. Fucking hell
Let’s be honest, with their attendance rate in class, I don’t think these students actually vote…
and they’ll be assured they’re deserving of a tech sector job while everyone else is already losing their everything.
As someone who works in ed tech these days, I’m kind of down for them as a study tool. For example, synthesizing notes and turning them into flashcards, practice tests, etc. I find that stuff to be suuuper handy if I’m trying to learn something.
But for cheating, yah, fuck that noise. A lot of classes are moving back to pencil and paper because of this, and I totally support that.
I feel like synthesising notes and turning them into flash cards how i learn things.
Exactly. Taking notes in class during a lecture. Copying something the instructor wrote on the board. This is all part of the learning process. The act of doing these things helps you learn.
The only skills or learnings I really seem to have retained from University are the ability to collect, and collate information and then apply it to a problem. The actual information collected and problems solved are lost to me now.
The good students are still getting A grades naturally. And the bad students are getting A grades with ChatGPT. A grades for everybody! (Until we get to the closed-book, in-person test at the end of the year…)
‘A’ grades are suddenly everywhere, as introduction of stochastic parrot as a service reveals the education system is geared towards training parrots instead of teaching humans. What a surprise.
Surely this won’t cause any problems at all
Shouldn’t the education system change then? If it is easy to get an A with a machine, should we then not focus on learning something that can’t be done by a machine. I mean, it has value to know things and know what to do insituations where AI is not available.
