Follow Slashdot blog updates by subscribing to our blog RSS feed

 



Forgot your password?
typodupeerror
Education Technology

Automated System Developed To Grade Student Essays 253

RougeFemme points out this story at the Times about software that can be used to grade student essays and offer almost instant feedback. "Imagine taking a college exam, and, instead of handing in a blue book and getting a grade from a professor a few weeks later, clicking the 'send' button when you are done and receiving a grade back instantly, your essay scored by a software program. And then, instead of being done with that exam, imagine that the system would immediately let you rewrite the test to try to improve your grade. EdX, the nonprofit enterprise founded by Harvard and the Massachusetts Institute of Technology to offer courses on the Internet, has just introduced such a system and will make its automated software available free on the Web to any institution that wants to use it. The software uses artificial intelligence to grade student essays and short written answers, freeing professors for other tasks."
This discussion has been archived. No new comments can be posted.

Automated System Developed To Grade Student Essays

Comments Filter:
  • This is horrid (Score:5, Insightful)

    by swm ( 171547 ) * <swmcd@world.std.com> on Thursday April 04, 2013 @07:19PM (#43364089) Homepage

    One of my kids had something like this: not for English, but for physics.
    The teacher couldn't be bothered to assign and grade proper homework.
    Instead, he fobbed the kids off onto a web app.
    - go to the site
    - get a problem
    - solve the problem
    - type in the numerical answer
    - right answer? go on to the next problem
    - wrong answer? try again
    The web app allowed maybe 0.5% margin for rounding error, and you got 5 tries before it failed you on that problem.

    It sounds reasonable in the abstract, but in practice it was utterly wretched.
    All learning is, at some level, an interaction--a conversation--between student and teacher.
    Even if it is nothing more than a red check mark or a red X on a homework paper,
    you have communicated some thing to some person and gotten some response.
    You don't realize how important this is until it is gone.

    With nothing but a machine to talk to, it stops being about learning.
    It is just about satisfying the machine by whatever means necessary.
    In his rage and frustration my son told me that the easiest way to solve the problems was to copy and paste the problem text in to google.
    This would reliably return the general formula for solving that problem;
    plugging in the numbers that the web app had generated for your instance of the problem would then yield the correct answer.
    By the end of the school year, I was telling him that if he didn't want to deal with the web app, he should use google to get his grade,
    and if he wanted to learn physics, I would teach it to him.

    Automated essay grading is going to be even worse.
    There is no point writing prose unless a human is going to read it.
    When I want to talk to machines, I write code.

    Writing songs, that voices never shared...
    -- Paul Simon

    • Re:This is horrid (Score:5, Interesting)

      by Anonymous Coward on Thursday April 04, 2013 @08:10PM (#43364537)

      I went through the same system and it taught me all sorts of useful things unrelated to my actual physics curriculum, like
      1/2 != 2/4
      0.5 != 1/2
      x != x+1-1
      x^2 != x*x

      • Re:This is horrid (Score:5, Interesting)

        by reve_etrange ( 2377702 ) on Thursday April 04, 2013 @08:13PM (#43364557)
        I had the same experience in university calculus and physics. Even for problems with one right answer, there are typically many (even infinite) ways of expressing that answer. Even something as advanced as Mathematica or Maple can be fooled, and the websites in question are no Mathematica.
        • Re:This is horrid (Score:5, Interesting)

          by Jstlook ( 1193309 ) on Friday April 05, 2013 @12:08AM (#43365699)
          I went through that type of system for a Chemistry class. After class, the entire class would wander down to the computer lab and do the homework together. We'd get a question, find the [book] answer, then have each person try to obtain the correct [computer-identified] solution on the first try, trying various syntax adjustments each time. Of the three chances we got, someone usually got the right syntax before everyone had failed the question the second time.

          Best benefit? Getting a group of people in the same place to research, debate, and agree on a single answer, then be open-minded and organized enough to shape the solution to fit the constraints given.
      • Maths is all about simplification. Examples 1, 3 & 4 are poor answers because they haven't been fully "simplyfied" (which is more than just a character count), example 2 doesn't have enough context but it could be "correct" in the context of a lesson about decimal points. OTOH I doubt that was the logic behind the software's answer.
    • There's nothing wrong with using automated marking where it's actually saving time, as long as the results are used to further learning. The simple idea (employed by the Khan academy and others) is to get students/pupils to practise at home, a system tracks what they can do and what they struggle with, and the teacher-pupil/student-professor contact time is used to address any issues. Grading simple, repetitive tasks is a waste of time if it can be automated; it's the feedback that matters. I assume that
    • Re:This is horrid (Score:4, Informative)

      by RougeFemme ( 2871421 ) on Thursday April 04, 2013 @08:33PM (#43364691)
      I'm currently tutoring my daughter in statistics for the same reason. She's in college and while she's flipping through her homework appliication and her e-textbook, I'm flipping through my old statistics books, plus a couple of study guides I picked up. Also, sometimes the homework application is simply wrong. (Doesn't every tool/program have at least one bug?) My sister, a teacher, uses one - mandated by the community college where she teaches. Occasionally, she has to override the application so that she can mark correct problems that the application marked wrong. The students alert her, she checks and then overrides when the application is clearly wrong.
    • by Roger W Moore ( 538166 ) on Thursday April 04, 2013 @08:40PM (#43364739) Journal

      It sounds reasonable in the abstract, but in practice it was utterly wretched.

      No, the abstract does not sound reasonable: as with most things online you can always find bad ways to do it. I'm a physics prof working as part of a team to develop an open source, algebra capable question and content system. However even the current capabilities of something like Moodle [moodle.org] (which is Open Source) is far in excess of what you describe. You can type in multiple "answers" to a problem and have the student get feedback and a partial grade if they get the problem wrong in a way that you managed to guess. Obviously if they find a new way to get it wrong then they will not get feedback though.

      Commercial systems go even further with the student having the option to click on a help button which can break the question into steps for the student to complete in rder to guide them through to the right answer. This can be configured to give a grade penalty at the choice of the instructor - this is one of the features we want to add to an Open Source solution.

      However even with current Moodle capabilities you can build a system that, I would argue, is better pedagogically for many physics problems (those with numerical or symbolic responses) than paper-graded assignments because, with an online system with some feedback and multiple attempts the student is encouraged to keep trying until they figure out how to get it right. This encourages them to think out the solution themselves whereas with a paper assignment they get one try and are then given the answer. To make this work though you need some means for students to come and talk to you and/or TAs to provide some help towards getting the right method. So you still need the student-teacher interaction but computers can provide a first line of contact and so let a teacher help more students.

      That being said I find it exceedingly unlikely that this EdX system can work for written responses beyond checking that their english is good. For physics how can it possibly know that the statement "the Higgs boson has a mass of 140 GeV/c2" is wrong and "Dark Matter does not interact with photons" is correct? To be able to grade it will have to know a huge amount of information about a massive range of topics - and looking this stuff up on Google is not an option given all the crazy people and their wacky physics theories which they stick on a web page.

    • That sounds like he has a pretty good teacher. If it were me, I'd be fine with the link Google produced. :P

    • In his rage and frustration my son told me that the easiest way to solve the problems was to copy and paste the problem text in to google. This would reliably return the general formula for solving that problem

      Isn't that what physics is? Applying the right formula? I don't think much is gained by having students memorize formulas anyway. In real life you just look up general formulas on google/wikipedia/wolfram and apply them to your specific problem. I suppose it's useful to be able to derive certain formulas as a method to gain greater insight into patterns and ways of thinking, but I don't think this depth is commonly required in high school.

      If it were my kid, I'd rather he knew how to google formulas and

      • Isn't that what physics is? Applying the right formula? I don't think much is gained by having students memorize formulas anyway. In real life you just look up general formulas on google/wikipedia/wolfram and apply them to your specific problem. I suppose it's useful to be able to derive certain formulas as a method to gain greater insight into patterns and ways of thinking, but I don't think this depth is commonly required in high school.

        If it were my kid, I'd rather he knew how to google formulas and apply them to specific problems than have a bunch of formulas memorized. People naturally memorize things if they use them enough. Rather than having kids memorize specific formulas, I think it's more useful that they memorize the best way to find specific formulas (i.e. google, wolfram, etc). That's much more efficient.

        I have a bunch of physics formulas memorized because I used them a lot. The ones I don't use a lot I just look up when they are needed. If I forget one, it's not a big deal. If I forget how to look up formulas (not sure how that would happen) I would be screwed.

        The purpose of EDUCATION is to teach the student how to understand and think about the material.

        • That's not the ultimate purpose. The ultimate purpose is to solve problems. Having a useful way to think about problems is a good tool for solving them. Afterall, that is the way we judge whether a particular way of thinking is valid or not, or whether someone really understands something. We judge understanding of a problem by whether someone can solve a similar problem. We judge whether a "way of thinking" is correct by whether it leads to correct solutions to problems.

          • Re:This is horrid (Score:4, Insightful)

            by Pulzar ( 81031 ) on Thursday April 04, 2013 @11:07PM (#43365421)

            That's not the ultimate purpose. The ultimate purpose is to solve problems.

            The *ultimate* purpose of education in science is to solve problems we have no current solutions for. They are not solved by looking up the formula, but by developing your own formula based on your understanding of how things work.

            I don't need to look up the formula that allows me to calculate the acceleration of a body of known mass when known force is applied to it, because I understand their relationship. I also understand the relationship between velocity, time, and acceleration, so I can create further formulas based on these two sets of relationships that might've not been obvious at first.

            If I've just looked up the final formula, I've skipped the important steps that give me the underlying understanding of physics, which will allow me to create new formulas to solve new problems.

    • by rsilvergun ( 571051 ) on Thursday April 04, 2013 @09:17PM (#43364937)
      Face it, we're all going to get replaced by Expert Systems. They talked about this in the 80s and you didn't believe. 95% of us follow pretty simple patterns. There's damn little that most of us can do that a machine can't. Sure, there are exceptions. But most of us don't qualify, we just think [xkcd.com] we do.

      You're being replaced. The real question is how are you going to deal with it? What do we do when 95% of us are completely unnecessary?
    • I've got that for Chemistry AND Physics. It's horrible. I've entirely missed problems because I couldn't figure out how it wanted the answer represented.

      I mirror your sentiment exactly:

      The teacher couldn't be bothered to assign and grade proper homework.

      The kicker is that teachers who DO assign real homework have TA's or graders to grade the homework, all they actually grade are our tests.

      Oh yea, and I have to pay extra for this online bullshit. It's required in a more in-depth way than textbooks (I literally can't pass the course without it) but isn't paid by the scho

    • After reading over thirty of the posts, many worthy, I return to my first conclusion after reading yours: you said enough to cover the absurdity of what eDx is trying to do. The only possible useful thing I could see for it at first blush would be to act as a rough filter for very basic errors - and it'd still be making mistakes. At a time when there are plenty of sufficiently literate people un- or under-employed, let them grade essays at least as a first filter. One needn't be an expert on a subject t

    • I graded undergrad essays of the children of privilege for well over a decade before progressing to their graduate essays.

      Overall, I don't think automated grading is going to make a difference. Since most of the students don't care, I can't imagine there's much harm in having a grader who doesn't care. The notion of "caring" in higher education has been made obsolete, along with live lecturers. And I don't really blame the students gpt mpy vstomh, because under the stress of the huge debt that our system

    • Your son learned something important: Rote learning is a thing of the past, and being able to know where to find the answer is more important than having it, and finding someone else to do the work for him is easier than doing it himself.

      He's going to do great in a management position.

    • by _KiTA_ ( 241027 )

      I think that particular app, which I forget the name of, is very popular nowadays in Intro Physics classes. It's outrageously annoying. Hilariously, all the answers and solutions guide are available via a quick google search, something my classmates were quick to tell the rest of us.

      The scary thing is, the people who make these books, who re-release the book every 6 months so you can't buy them used, are trying to push for the same kind of automation... of lectures. You'll have a "teacher," but they won't b

    • by tlhIngan ( 30335 )

      My discrete math course did that - it had a bunch of web pages and tools you need to use to solve the problems. They never worked right at all.

      OTOH, I can see the system being used in those massively online open courses (MOOCs) where a lecture can admit easily 3000+ or more students in a term. This seems like a way for them to have homework in the course and to do tests so it closes the loop. The prof then gets back a summary of how many people get a question wrong, etc.

      So it has its uses and if we're goin

  • by gewalker ( 57809 ) <Gary.WalkerNO@SPAMAstraDigital.com> on Thursday April 04, 2013 @07:21PM (#43364107)

    Take one lab report for Fluid Mechanics, measure the thickness with a micrometer -- look up the grade on the curve.

    • Did he at least flip through to make sure there weren't a bunch of blank pages in the middle?
    • We already have that today, you can guess any of the AP/SAT/GRE essays' grades with phenomenal accuracy just by looking at how long they are:
      http://www.nytimes.com/2005/05/04/education/04education.html?_r=0 [nytimes.com]

      • by demonlapin ( 527802 ) on Thursday April 04, 2013 @09:43PM (#43365073) Homepage Journal
        That's interesting in its own way, but a much more interesting comparison would be between the essays' lengths and the respective SAT Verbal scores of their writers. I would bet that they are also correlated quite closely.

        News flash: when presented with an essay topic, smart people spend a few minutes planning and then proceed to write voluminously about the subject, because they are fluent writers. Dumb people start muddling along, lose track of where they are, and stop when they've stated (though not proved) their main point, because they're not. Fun game: ask a room full of people to write nonstop for five minutes on any topic(s) of their choosing, then compare word counts vs IQ/class grades/whatever.

        If you're a HS student reading this (and I imagine there are a lot of you who are): practice writing. Practice writing. Practice writing. It's important. It's probably the most valuable skill you will ever acquire for dealing with people you don't meet with face-to-face. Bad writing is universally considered a sign of low intelligence. It takes a lot to overcome the negative impression that bad writing gives, and you often will not have the opportunity to try - when given a stack of 100 resumes for two positions, guess how the initial winnowing occurs? Toss anything on colored paper, anything written in a funny typeface, and anything with grammatical or spelling errors. I cringe today when I read some of the stuff that I wrote in HS, but it's grammatical and correctly spelled, even if the verbiage is ponderous (and occasionally verges on purple prose).
        • by DragonWriter ( 970822 ) on Friday April 05, 2013 @12:28AM (#43365791)

          News flash: when presented with an essay topic, smart people spend a few minutes planning and then proceed to write voluminously about the subject, because they are fluent writers. Dumb people start muddling along, lose track of where they are, and stop when they've stated (though not proved) their main point, because they're not.

          IME, smart people write concisely and to the point of the prompt, while dumb people write voluminous, rambling, redundant, and unfocussed walls of text.

          "Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away." --Antoine de Saint-Exupery

    • Take one lab report for Fluid Mechanics, measure the thickness with a micrometer -- look up the grade on the curve.

      I guess that beats throwing them down a flight of ten stairs twice.

  • by hawguy ( 1600213 ) on Thursday April 04, 2013 @07:27PM (#43364161)

    Seems like it's a small step from this to having computer algorithms that automatically write your paper for you too - then you can let it go through thousands of submit-edit-submit cycles until the scoring computer gives you a perfect score.

    Kind of like the guys that came up with software to generate nonsense scientific papers and actually had a few accepted [wikipedia.org] at conferences and journals.

    • by korgitser ( 1809018 ) on Thursday April 04, 2013 @07:32PM (#43364207)

      I wonder how these would do:

      the postmodernism generator http://www.elsewhere.org/pomo/ [elsewhere.org]
      the math paper generator http://thatsmathematics.com/mathgen/ [thatsmathematics.com]

    • by lgw ( 121541 )

      Much of modern finance actually works this way. Press releases about mergers and such are written by computer to be read by computer and fed into algorithmic trading. The relationship to nonsense is left as an exercise for the student's computer.

      • Oh, that's it! I was wondering why I don't understand it, but it's actually not in a human readable format, it just disguises as English.

    • by fermion ( 181285 )
      This pretty much already happens. The question is are you in school to learn or to get a grade.

      One problem with teachers grading papers in today world is that you have no idea if the work is original. Reports for all the common topics are all over the place, and it is easy to adjust and mix and match so the common tools will only catch the laziest students. Solution to textbook problems are all over the place as well. The is why any decent science or math teacher will give computer generated tests. A

  • by milbournosphere ( 1273186 ) on Thursday April 04, 2013 @07:29PM (#43364179)
    Why I want to goto Harvard By P Q Student Up up down down left right left right B A
  • feedback... (Score:5, Insightful)

    by retchdog ( 1319261 ) on Thursday April 04, 2013 @07:30PM (#43364191) Journal

    ``Your grade is C. To improve your grade in the future, you need to do the following:

    use 25-30 words per sentence; include more words from the wordnet entry for the topic of your essay; avoid simplistic or run-on sentences as measured by number of noun and verb phrases detected by our proprietary NLP tokenizer.

    As a helpful reminder, our preparatory guides are available as a subscription service and include 100 practice submissions per week; only $29.95 per month."

    • The sad thing is that's how ETS already grades essay questions, only the "tokenizer" is an underpaid college graduate.
    • by JMZero ( 449047 )

      If their use was kept secret, systems like this would likely perform well most of the time; the correlations these systems are based on are probably pretty steady.

      Once students get any information about the system, however, it's doomed - and in any case it's unlikely the system will give real, useful assistance in improving skills beyond what you'd get from a grammar checker.

  • by FailedTheTuringTest ( 937776 ) on Thursday April 04, 2013 @07:30PM (#43364193)

    Grading is not, or should not be, about the grade, it should be about the feedback that the lecturer gives to the student. Even if the computer can grade an essay well (which I remain to be convinced of, although I am sure I will soon have the chance to test it for myself), there is no claim made about the computer giving useful advice to the student. Can a computer explain how to refine a research question or structure an argument? Sadly, many lecturers don't in fact give good feedback, but we should be looking for ways to enable lecturers to give better feedback, not accepting poor feedback as the norm.

    • Even if the computer can grade an essay well (which I remain to be convinced of, although I am sure I will soon have the chance to test it for myself), there is no claim made about the computer giving useful advice to the student.

      Since being able to grade well requires the ability to make the exact same distinctions required to identify the feedback that would need to be given, I would be very surprised if software that could do one could not also do the other.

      I'd also be surprised if current software was

    • Grading is about the grade. Learning is about the feedback. Unfortunately, more and more, the educational experience is about the grade or standardized test score rather than learning. . .and learning to love learning. . .and learning how to learn. Kids don't have to show how their work in math anymore; all the teachers care about is the answer. We shouldn't be surprised at this latest development - well, not too terribly surprised.
    • Grading is not, or should not be, about the grade, it should be about the feedback that the lecturer gives to the student.

      Not always: there are two types of assessment [learnalberta.ca]. Formative assessment where the aim is to let the student know what they understand and what they need to work on. This is what you describe. There is also Summative Assessment where the aim to to assess what the student actually knows. Usually I try to get some of both - for example although a midterm is mainly aimed and finding out how much the students have learnt I'll also spend a lecture to go through the exam to give detailed solutions and feedback to stu

    • I suspect that these automatic graders would give the same grade to an essay if all the sentences were rearranged, or if the nouns were randomly switched around.

      For example, the previous sentence might receive the same grade as this one:

      Sentences suspect that these automatic nouns would give the same essay to a grade if all the graders were rearranged, or if I were randomly switched around.

      In my defense, I fixed the articles to make the rearranged sentence conventional in terms of article use. But that c

    • by clifyt ( 11768 )

      "Grading is not, or should not be, about the grade"

      I worked with Dr. Shermis back in the mid '90s on this (one of the professors quoted in the article) and we had our own software we were working on.

      One of the very first things we had agreed upon was that grading and rating are two very different things. And when we worked on our software, it was designed to give back several scores that summarized why there was an overall score that was given.

      Beyond this, *MOST* work in this area are not based around how

  • Watch as we move from "search engine optimization" to "grading engine optimization," as students look for AI solutions to write their papers, freeing them for other tasks.
  • by GoodNewsJimDotCom ( 2244874 ) on Thursday April 04, 2013 @07:34PM (#43364239)
    My friend wrote a story about his cat that was grammatically correct,and used big words, but made little to no sense. The auto-grader program told him he was approaching PHD level English. So he took his paper into school and showed it to the English teachers who reviled at it. He was like,"Show's what you know, the computer told me I'm university level."
    • While I am rather skeptical about the quality of AI essay grading, I'd be very surprised if software in this area hasn't advanced since 1991. I mean, software in most other domains certainly has.

      • One thing is for certain, it won't be able to understand context of the article. For AI to understand what you're talking about, it basically needs to be able to imagine it. For a computer to imagine stuff, it'd need a 3d model of the situation and a huge library of nouns, verbs, adjectives, adverbs. The day we have true AI, I might think we'll have the capabilities of natural language interpretation. Until then, the best it can do is weigh sentence structure "quality".
        • IBM's Watson is capable of these tasks as demonstrated by the Jepordy stunt. Of course Watson is one of a kind and requires 20 tons of air conditioning equipment, but it is doable. The most interesting part of the stunt was when they gave Watson the ability to "hear" the human answers (both right and wrong), just that one bit of information basically wiped out all the wrong answers that arose from Watson not "understanding" the category.
      • by Ksevio ( 865461 )
        I know MS Word has a simple analysis that can tell you stuff like that. WordPerfect probably had something similar.
  • by skywire ( 469351 ) * on Thursday April 04, 2013 @07:35PM (#43364243)

    Every era has its snake-oil salesmen and their marks. Sadly, in this case it will not be the customers who suffer, but their hapless students.

  • by doug141 ( 863552 ) on Thursday April 04, 2013 @07:38PM (#43364275)
    "A director of writing at MIT Les Perelman says that because these robo-graders work according to an algorithm, it is not hard to find out what it values and thus beat the system. He found that if you write long essays with big words, even if they are nonsensical, you will score high. The algorithm does not like short sentences or paragraphs or sentences that begin with ‘and’ or ‘or’ nor is it enamored of sentence fragments. In other words, all the little rules that good writers will break to create a particular effect will cause your essay to be marked down.

    Perelman gives an example of how you can get a high score. The most interesting feature of the algorithm is that it doesn’t care about substance or even truth. It will ignore such trivialities as saying that the war of 1812 began in 1945, provided you say it grammatically. The substance of an argument doesn’t matter, he said, as long as it looks to the computer as if it’s nicely argued.

    For a question asking students to discuss why college costs are so high, Mr. Perelman wrote that the No. 1 reason is excessive pay for greedy teaching assistants. “The average teaching assistant makes six times as much money as college presidents,” he wrote. “In addition, they often receive a plethora of extra benefits such as private jets, vacations in the south seas, starring roles in motion pictures.”

    E-Rater gave him a [top score of] 6. He tossed in a line from Allen Ginsberg’s “Howl,” just to see if he could get away with it. He could."

    http://freethoughtblogs.com/singham/2012/05/03/how-to-fool-a-computer-grader/ [freethoughtblogs.com]

    • But this is precisely why it makes this system useful AND should please teachers. If the system works well for grammar and ( hopefully programmable) essay structure, then the teachers can focus on the content, style, and finer points of writing. A computer can correct to/too/two, and if it frees up more time for the expert that is being paid to grade in depth, this is a good thing.
    • Reportedly, long essays with big words also help your SAT writing score. And in my state - which shall remain nameless - it definitely seems to help on the standardized writing tests.
    • So all you really need is x (where x matches the length of the assignment) occurrences of the word buffalo, with the first one capitalized and the last followed by a period. A grammatically perfect sentence, with the maximum possible length results.

    • Get ready to see SEO (Submission Engine Optimisation) specialists cropping up and making a fortune off lazy students.

      Or should that be ESO (Essay Submission Optimisation)

      And new websites such as LOLEssays where you can see examples of the most ridiculous essays people actually got top marks for.

  • Any algorithm can be beat
    The all you have to do is fill your paper full of the right keywords

  • No matter how sophisticated the algorithm, the set of strings that get graded an A is bound to contain some weird and illegible elements. They probably won't be too hard to find by inspection of the algorithm and its training data. It will only take a few widely publicized examples of meaningless essays with a high auto-grade to cast doubts on this method of grading, no matter how effective it is in the common case.
  • The software uses artificial intelligence to grade student essays and short written answers, freeing professors for other tasks."

    There are probably at least several good jabs to make at this so I'll try to just address the best one.

    First thing that comes to my mind is 'free them up to do what?" Education is adding more and more distance between the student and the teacher without throwing this into the mix. In a perfect world, teachers would be there to teach and that's it. But it's become more a problem

    • Get me back to the good 'ol days where your prof knew your name.

      That would be nice. But, of course, back in those days a person like me wouldn't have had a shot at higher education, much less a Ph.D. No, I'd probably be on my way to black lung or some similar ailment by now, working in mines acquired by some guy who could afford to send his kids to a low student-to-teach ratio private school. The democratization of education has its costs as well as its benefits.

  • Would you like fries with that?

    Seriously though I don't think writing to what an algorithm wants is a bad thing if the algorithm wants the right stuff.

    It's not as if students don't write to the algorithm the professor uses even now. The only difference is where the algorithm is stored and how flexible it is.

  • At my university we have AI that continually checks our essays as we write them. It even points out the specific mistakes, gives suggestions to fix them, and allows us to rewrite that section of the essay.

    We call this advanced AI MS Word.

  • So basically they feed the essay into Google spellchecker and count the number of underlined words?

  • How many times?

    I can envision an essay writing 'bot making multiple submissions with some AI to optimize subsequent submissions based on the .

    I know it sounds like an awful lot of effort to build, but its time better spent than writing a two page essay on Dickens, IMO.

  • This is now the n-th time something like this has been on /.

    The answer is still the same: This is a very bad idea. Students will learn how to game the software instead of how to write well. No software can grade whether the reasoning is sound, the images vivid, the prose well readable. But what students will learn is that writing essays is not important, after all it is not even worth bothering a human being to read and grade.

    • Now assume you grow up somewhere where the teachers can't read and write proper English. We can call this place "public school in a non-educated community unless you're lucky." This kind of thing could be useful. The only problem is that people would not be smart enough to correct it, but it might still raise the bar.

  • by msobkow ( 48369 ) on Thursday April 04, 2013 @08:45PM (#43364765) Homepage Journal

    Any professors I've ever known or been taught by had their grad students doing the grading, anyhow.

    Besides, what exactly are the professors being "freed up" for? Isn't their JOB to TEACH?

    Oh, yeah, I'm thinking old school. Nowadays a professor's job is to find corporate grants...

    • Depends on the school. Good, small undergrad schools use little or no TA grading.

    • by supercrisp ( 936036 ) on Friday April 05, 2013 @08:09AM (#43367257)
      "Isn't their JOB to TEACH?" Not completely, sometimes barely at all. At an R1, the typical humanities appointment is 25-40% teaching, 50% research, and the balance to service. Some faculty may only teach one class a semester, if they're administrating a department or subdivision of a department, or if they're running a onerous committee, like a hiring committee. At a teaching school, your "main" job is teaching, but you're still required to produce some token level of research and serve the university in other ways, such as by working on committees, being a public figure, and other stuff that you might not consider right away. So, at my job, at a teaching school, about 70% of my time goes into teaching. The rest goes into mandatory requirements to publish, present papers, do committee work, assist developing colleagues, and perform community service. (Note that in my annual performance review, I'm only allowed to indicate that teaching was a maximum of 60% of my effort, and this at a teaching school. This may be atypical, but I suspect it's not.) Now, in the sciences there are faculty with no teaching requirements. And in the humanities, at R1 schools, faculty get a year or a semester off periodically during which time they are expected to complete a research project, typically a book.
  • Good for the filler / big lecture classes to move to a full online / test only setting and maybe they can pass on the lower costs to the students.

    It may just lead to people to gameing the system but what does that give them? More time to work on there core classes?? I think that it is better to say do the min to pass art history so I have the time to work on the classes I want to work on.

  • by RobertinXinyang ( 1001181 ) on Thursday April 04, 2013 @09:01PM (#43364835)

    I read the article and went to edX. At edX I signed up; but, I can not find out about this system. Quite Frankly, I am a teacher and I need my students to be writing more; however, I do not have the time to grade all of their papers so I have been assigning more objective homework that I would like.

    A system like this may work as a first pass filter to do the bulk of the grading, allowing me to focus on identifying common problems and developing lessons based on common errors rather than tying myself down with a huge stack of papers. This would also benefit the students by providing them with more consistent grading and feedback.

    This may not be what I go with; but, I would like to have a look at it. That takes me back to my question, can anyone point me to somewhere that I can get more information on this?

    • I read the article and went to edX. At edX I signed up; (semicolon should be a comma) but, (misplaced comma) I can not (should be "cannot") find out about this system. Quite Frankly,(lower case!) I am a teacher and I need my students to be writing more; (Run-on sentence. Use a period.) however, I do not have the time to grade all of their papers (comma here) so I have been assigning more objective homework that I would like.

      Grade: D

      • If I really care about what I am typing, I type it in word first. My students know I am the worlds worst typist when I type on the overhead; that is why I make an effort to use class material that I have written in advance. If I take the time to write well, which I am definitely not doing now, I can write, and have written, material suitable for publication

        I will also add that I am a business teacher, I wrote well enough to make it through an MBA program (oddly enough, I was even called on to help edit for

  • side effect of big classes. also can drop tests down to a Multiple hidden choice test. It's like a Multiple choice test without knowing what the choices are.

    smaller classes and team based work maybe even apprenticeships / more of a tech / trades schools level testing where it's more about real skills and not about test cramming taken to the next level where you now just need to know the buzz words.

  • And this is what higher tuition is buying?
  • aka, "your lazy bum grades me with a computer, my lazy bum lets a computer write one".

    How long until such a system is being gamed? Quite frankly, I wouldn't be too surprised if within weeks, students notice what the computer "wants" to "read" and write their essays accordingly.

    Not that it would change too much from today, where you learn what your professor wants to read and write for your audience...

  • by milkasing ( 857326 ) on Friday April 05, 2013 @12:30AM (#43365809)
    All your GRE essays are evaluated by a machine and have been for years -- the e-rater. http://www.ets.org/research/topics/as_nlp/writing_quality/ [ets.org]
    The rating is also done by humans. It works well in practice and ensures that essays are graded fairly. If there is a significant discrepancy between the two ratings for a essay, that essay is examined further by another specialist. It prevents students from being victims of someone having a bad day at the office, and also does not encourage writing an essay to beat a machine.
    The significance of the EDX news is not the concept of automated grading, it is that that such software is now free and opensource.
  • with the prevalence of advertising and corporations dominating everything, you can be sure that soon the answer to all questions will be "Pepsi"

    http://homepage.smc.edu/nestler_andrew/pepsi.jpg [smc.edu]

  • I've probably been in this longer than anyone - in 1986 I was working with a teacher (High School Biology) who had networked C-64s in his classroom. Of course back then the questions were all multiple choice (we couldn't give it enough intelligence to evaluate expressions), and yes he did the semester tests himself.

    If used properly, there is nothing especially wrong with doing assignments or quizzes on computer. That being said, you know there is going to be a tendency to misuse them. They'll assign more wo

Marvelous! The super-user's going to boot me! What a finely tuned response to the situation!

Working...