AI Boosted Homework Scores, Then Exam Scores Dropped (economist.com) 53
An anonymous reader quotes a report from The Economist: Students and school children are increasingly using artificial intelligence. A survey last year by Chegg, an ed-tech firm, found that 80% of rich-world undergraduates used it in their studies. More recent polls put the figure at 94% in Britain and 93% in Germany. Such widespread adoption has fueled concerns about the impact the technology might be having on learning. Teachers report marking identikit essays that they suspect are churned out by ChatGPT. Yet firm evidence on AI's educational effects has been lacking.
David Stromberg of Stockholm University and Victor Lei and Wu Yanhui of the University of Hong Kong set out to fill the gap. They tracked 27,000 pupils aged 12-18 in China, where AI adoption has been fast. Around 80% reported using models such as Doubao and DeepSeek; the other 20% formed the control group.
The results are eye-opening. After six months, pupils using AI saw their average homework score rise by 18% across all subjects. The time they took to complete each assignment fell from an average of 64 minutes to 45. But come exam time, the same students scored 20% below their classmates who had not called on AI's help. Homework scores once predicted exam performance; now those who score highest are, perversely, more likely to do worse in exams. The takeaway: "AI can boost learning productivity, but only for those who use the technology intelligently," reports The Economist. "Students must resist the temptation to reach for an AI-generated answer before thinking things through for themselves."
David Stromberg of Stockholm University and Victor Lei and Wu Yanhui of the University of Hong Kong set out to fill the gap. They tracked 27,000 pupils aged 12-18 in China, where AI adoption has been fast. Around 80% reported using models such as Doubao and DeepSeek; the other 20% formed the control group.
The results are eye-opening. After six months, pupils using AI saw their average homework score rise by 18% across all subjects. The time they took to complete each assignment fell from an average of 64 minutes to 45. But come exam time, the same students scored 20% below their classmates who had not called on AI's help. Homework scores once predicted exam performance; now those who score highest are, perversely, more likely to do worse in exams. The takeaway: "AI can boost learning productivity, but only for those who use the technology intelligently," reports The Economist. "Students must resist the temptation to reach for an AI-generated answer before thinking things through for themselves."
Get rid of homework (Score:5, Insightful)
Re: (Score:2)
Homework never works at home.
Re: (Score:2)
100% this. Even a short 5-minute quiz with just a couple questions at the start of each day will do the trick. But you simply can't rely on homework scores to be a proxy of learning or time spent studying anymore. Brief daily quizzes on the previous day's material will force them to.
Also, It's not like students have the excuse of, "I couldn't study, I didn't have anyone to explain X to me" anymore. The very AI that they might otherwise use to cheat on their homework can also explain to them the things th
Re:Get rid of homework (Score:4, Interesting)
But you simply can't rely on homework scores to be a proxy of learning or time spent studying anymore.
You never could. Parents do their kids' homework all the time, or siblings, or friends. Ideally, teachers should give a large pile of optional homework. State right up front that although they will grade it, it will not count towards your final grade, and the grade on each homework paper will be given as a percentage of the problems attempted, not of the total set of problems. So if you do five problems and get them all right, you get 100, and if you do 50 problems and get most of them right, you get 90. At that point, the purpose of homework is primarily to help the students learn, rather than being primarily a way to evaluate the students. Give some participation points for doing some of the homework, but not based on correctness.
This has three benefits. First, the smart kids who understand the material quickly will do just enough to make sure they understand it and then will stop wasting their time, but will still have feedback on the work that they did do. Second, the teachers won't have to grade as much work from students who don't really need that much extra work. Third, no at-home cheating, whether by technology or by other people will distort their grades.
Obviously this doesn't work for everything — essays, for example — but for general homework, it's a big win.
Brief daily quizzes on the previous day's material will force them to.
You can certainly do that along with what I suggested, though I'm not sure how much difference it will make in the grand scheme of things.
Also, It's not like students have the excuse of, "I couldn't study, I didn't have anyone to explain X to me" anymore. The very AI that they might otherwise use to cheat on their homework can also explain to them the things they missed in last year's algebra class that are blocking them from learning this year's calculus, or whatnot. Congrats - you now have an infinitely patient personal tutor who will spend hours dumbing things down for you until you get it. Now you have no excuse - learn the material. It becomes all about actually putting the time in.
Agreed.
Re: (Score:2)
Re: (Score:2)
First, the smart kids who understand the material quickly will do just enough to make sure they understand it and then will stop wasting their time, but will still have feedback on the work that they did do.
I love that you didn't explain what the effect of this will be on the "not smart kids." Do you think they're going to hit the books and do a bunch of non-optional homework to learn the material? Have you met a single child?
Re: Get rid of homework (Score:2)
Re: (Score:1)
Homework has never been a productive use of the students time. If you are at school, you should be learning there. If you are at home you should be socializing with your family or friends.
Like think about it for 5 seconds. Why are people so angry now. Why are people addicted to social media? Because they have not learned boundaries.
If students are spending all their waking time doing homework for 8 different classes, with teachers who don't talk to each other to talk about homework load, you're going to hav
Re:Get rid of homework (Score:5, Insightful)
Most subjects are best learned through repetition. Learning your multiplication tables isn't something you just reason out. Learning how to do the process of long division or add fractions is best done by having the student do it thousands of times. Our little monkey brains need to be force fed information in order to build up any hope of retention.
People are addicted to social media because we adults shifted the responsibility for self-discipline onto children. That's bad, we are bad for letting it get this far, and we should feel bad.
Re: (Score:2)
This comment is a real head scratcher to me. So off the mark I blurted out a laugh when I saw it.
I'm not aware of any technology that can build a society where adults hold literacy and numeracy, that does not involve a lot of repetitive practice and rote learning. Even numerical theory as presented in common core mathematics (which I applaud) requires practice and repetition to retain. And the exercises in common core that Internet loves to complain about is an example of that practice.
Re: (Score:2)
Re: (Score:2)
I don't know where you went to college.. but for me, homework was where ALL my learning happened. Listening is passive. To learn the brain must engage.
How much programming did you learn by listening? Or did you learn by doing?
Math is the same.
As for "grades" people confuse why homework is graded in the first place. The main purpose is just to get you to do it. Nobody will do ungraded homework. So it's a carrot and the stick thing so that you do "your workout".
Re: (Score:2)
Re: (Score:2)
We made homework a completion grade only and provide students with the answer key. They can download a version of the homework with all the problems worked out.
The results?
Those who do the homework because they understand the value of repetition, do well on the quizzes and tests. Those who just copy the solutions... fail. In the four years we have tried this model, we have not had a single student who completed all the homework fail the course and we consistently score very well on standardized tests.
How do
Re: (Score:2)
Like think about it for 5 seconds. Why are people so angry now. Why are people addicted to social media? Because they have not learned boundaries.
You make a good point about how things are different "now" vs years ago -- the recent invention of ...homework.
Maybe you should consider thinking about thinks for more than 5 seconds before sharing your dumbfuck thoughts with the world
We have known homework is useless for years (Score:3, Interesting)
From what I heard the local chamber of commerce AKA a bunch of asshole rich businessmen who never worked a day in their fucking lives, lobbied the school board to drastically increase t
Re: (Score:2)
Re: (Score:2)
Agreed. It's a very expensive way to replace homework.
The CORRECT approach is to make homework voluntary, with no partial credit. Then do in-person oral examinations but only AT THE END OF THE DEGREE (ie the last week of a multi-year schooling process).
This used to be the case in universities and schools for centuries, and was replaced more recently by continuous assessments. Let's go back to the previously proven methods.
Psychologically, it's tougher on students, which is a good thing as it produces m
Re: (Score:2)
You're missing the point. Teachers can do this and make students fail in droves. But what teachers want is for students to LEARN, not to fail. The purpose of homework is not grades, but learning. The score is only there to motivate the student to actually do it.
What teachers need: an AI tutor. The AI tutor talks to the student, helps them with difficult stuff and gives them points as the students' interactions demonstrate knowledge of the subject matter. Then that becomes the homework. And the tests can be
Re: (Score:2)
Grades increase extrinsic motivation, but they often hurt intrinsic motivation. I've been thinking that we should replace grades with a single percentage number per subject. This number would tell that you have learned for example 12% of math. This percentage would be calculated automatically based on your answers on controlled environment (tests or class room work). Behind this number would be actual data that shows exactly the things you have not yet learned, so teaching could be targeted there. But as te
Remember why homework exists. (Score:2)
and test in-person. Your electronic devices go in a locker before entering the test area (many industries have this requirement).
And for the professional wanting to become the expert in their field, do you give the same advice?
A master doesn't get to 10,000 hours with only exams.
No. We shouldn't get rid of practice. We should remember the purpose of homework instead, because the exam results are proving it.
Re: (Score:2)
Shocked! (Score:5, Insightful)
Well, not that shocked.
Using an AI to do your homework gets the homework done without you learning the subject. Then you fail the exam.
Yeah, not shocked at all.
Re: (Score:3)
Yeah. There's been widespread misrepresentation of this (preprint!) in the media. What they actually found: test scores correspond to how much time you spend doing homework, regardless of whether you use AI or not. Which should not be a surprising result! Of course, the AI people, esp. those who had been using AI for more than a few months, stopped spending much time on their homework, and then their test scores fell to match the non-AI users who didn't spend much time in their homework either.
The TL/DR
Re: (Score:3)
No. having someone do the work for you absolutely reduces your learning of the homework material. it also reduces your time spent but your red herring is glowing nice and bright.
or knows the material
um, what? A test is absolutely on how well someone knows the material.
Playing devil's advocate today?
Re:Shocked! (Score:4, Interesting)
That is not what they found. What they found is that if you spend 60 minutes doing the homework with AI, you learn approximately as much as someone who spends 60 minutes doing it without AI. If you spend 20 minutes doing it with AI, you learn as much as someone who spends 20 minutes doing it without AI. Etc. The only real difference is that the AI people get way better homework scores, and as a result, tend to stop spending as much time, and thus tend to learn less.
That's what this preprint study found.
Re: (Score:2)
There are two types of students using AI.
One is a highly motivated intelligent student who is using AI to check and verify their work, and potentially challenge themselves with new problems if they wish to extend their learning session. That student, would likely score the same on homework and exams in 2026 as they did in 1986.
The other type of student, knows how to cut and paste homework into AI to get an answer. And the exam scores prove how many more of them there are. So do HR turnover rates for Gen
Re: (Score:3)
having someone do the work for you absolutely reduces your learning of the homework material. it also reduces your time spent but your red herring is glowing nice and bright.
He's got a valid point. For example, if you use wolfram alpha after you do a math problem to check the result you came up with you can actually learn MORE than just doing the problem yourself, and may even add to the time the work takes, as if your result is wrong, you'll spend extra time working out where you went wrong and fixing it. And ultimately learning more in the process.
OTOH - If you use wolfram alpha to DO the question for you, you will learn nothing at all.
But further to that, you can even use to
Re: (Score:3)
Yes, we should all be surprised that a tool created to increase efficiency at which documentation can be produced with little human input, indeed increases the efficiency at which documentation can be produced with little human input.
And when you take the tool away, the human that only knows how to use the tool is as useless as a broken tool.
Re: (Score:2)
I mean, kids used to learn how to extract a square root with pen and paper, then calculators came, now we wouldn't ask in a test how to extract a square root any more. New tools mean we learn different things and the content of tests and exams changes. To be very clear, I'm not saying "this is fine". But I don't think that the general argument that students lose certain knowledge due to a new tool is necessarily cause for alarm, as long as it also creates opportunities to learn new things.
Re: Shocked! (Score:2)
In other news, they found out that if you bring a forklift to the gym you lift a lot heavier weights, but build less muscle...
Not the end of the world (Score:3)
AI is just a fast information processor. The question is - is the end user a lazy, shortcut taking asshole with no work ethic because of lack of proper parenting or are they a driven worker that wants to learn faster.
Re: (Score:2)
The question is - is the end user a lazy, shortcut taking asshole with no work ethic because of lack of proper parenting
I obviously can't speak to your parenting.
You don't say... (Score:2)
This is great news (Score:2)
Exams should be hand-written on the spot. (Score:2)
Camera proctoring can reduce cheating.
Go ahead ... (Score:2)
Can AI be of use in learning? (Score:2)
Most of the comments to this story seem to be against using AI for homework, saying that it is obvious that this sort of use of AI results in worse exam scores.
But is it not also possible to use AI in a positive way for learning? There may certainly be some (many?) students who sabotage their own learning by getting AI to do their homework for them. But I suspect that some students, faced with a homework assignment, need extra tutoring before they have the skills to complete the assignment. Is AI not a good
Re: Can AI be of use in learning? (Score:2)
Re: Can AI be of use in learning? (Score:2)
Re: Can AI be of use in learning? (Score:2)
When Teachers, Need The Education. (Score:2)
New Title: AI Boosted Cheating, Then Exam Scores Dropped.
Apparently we have a bit too many edumucators that No One Left Behind, who can't seem to figure out why exam scores dropped. Especially after they made the grade of "F" illegal because of racism and got rid of SAT requirements for college.
Atrophy (Score:2)
People taking the elevator now ascend storeys faster than ever before!
But now they get winded climbing stairs, or walking across the length of the campus.
Rich-world? (Score:2)