Computers To Mark English Essays

digitig writes "According to The Guardian, computers are to be used in the UK to mark English examination essays. 'Pearson, the American-based parent company of Edexcel, is to use computers to "read" and assess essays for international English tests in a move that has fueled speculation that GCSEs and A-levels will be next. ... Pearson claims this will be more accurate than human marking.' Can computers now understand all the subtle nuances of language, or are people going to have to learn an especially bland form of English to pass exams?"

Graduate Record Exam

[ub3r n3u7r4l1st]

The GRE Writing portion is already using it.

From http://www.ets.org/portal/site/ets/menuitem.1488512ecfd5b8849a77b13bc3921509/?vgnextoid=ebd42d3631df4010VgnVCM10000022f95190RCRD&vgnextchannel=54c846f1674f4010VgnVCM10000022f95190RCRD [ets.org]

"For the computer-based Analytical Writing section, each essay receives a score from at least one trained reader, using a six-point holistic scale. In holistic scoring, readers are trained to assign scores on the basis of the overall quality of an essay in response to the assigned task. The essay score is then reviewed by e-rater, a computerized program developed by ETS, which is being used to monitor the human reader. If the e-rater evaluation and the human score agree, the human score is used as the final score. If they disagree by a certain amount, a second human score is obtained, and the final score is the average of the two human scores."

If you find a way on what the algorithm look for, even a software-generated essay can get 6's.

Re:Graduate Record Exam

[XopherMV]

This article isn't anything new. The GMAT already has a computer ranking the written assessment section of their test. Supposedly, it checks "over 50 structural and linguistic aspects, such as idea organization, syntactic variety, and subject analysis."

http://www.cybergmat.com/en/GMAT_Scores [cybergmat.com]

Re:Don't they already do this?

[jonadab]

> As a writing instructor, let me put it this way: I very,
> very seldom see a paper with misspellings and grammar
> mistakes that is nonetheless a well-written paper. It
> happens, but not often.

It happens most often when the writer is not a native speaker of the language. They'll write an essentially sound paper but make weird and obvious mistakes, like using the wrong preposition or spelling ph words with f. Depending on their native language they may also make other kinds of mistakes, e.g., Japanese people will frequently mess up grammatical number.

But the other poster may have been talking about grammatical structures that are actually a regular part of English grammar but are nonetheless consistently marked down by many English teachers, for obscure reasons. Examples of this kind of thing include split infinitives, the second-person imperative, the use of the second person pronoun to refer to anyone in general, and the use of objective-case pronoun forms in the predicate after certain verbs (particularly being verbs). Linguistically speaking these aren't actually mistakes as such, and in fact some of the contortions used to avoid them actively impede clarity, but they frequently get marked as "mistakes" nonetheless.

Re:Context...

[Ronald Dumsfeld]

The correct quote is, "Time flies like an arrow, fruit flies like a banana."

Re:Judging from...

[TheRaven64]

Pearson, the parent company of Edexcel is also the parent company of my publisher. They have just paid a human to proofread (all 950 pages of) my most recent book. A few things even the human had problems with, such as when one term should be one or two words, which depended highly on the context on which the word was used (not something simple, like whether it is a noun or an adjective). You'd think that, if they had an algorithm that was accurate enough to judge the quality of English then it would also be used for proofreading, but apparently not.

Re:Don't they already do this?

[digitig]

Indeed the rules of grammar can be seem obscure and almost arbitrary. However the rules of grammar8 actually grew naturally (i.e. not via committee, despite appearances) from a need of educated people to greatly clarify their communication.

Partly, but not entirely. There was a deliberate move in the 19th century to rid English of all those nasty Germanic influences and arbitrarily impose grammatical rules from the classical language onto English. The reason was nothing more nor less than intellectual snobbery, and the result was rules like not splitting infinitives and not ending sentences with prepositions. Those rules have no natural place in English; they were only put there to marginalise those who did not have a classical education.

Re:Graduate Record Exam

[mhelander]

Um, you should google that. Current consensus, I believe, is that his German was fine and that the donut in question isn't even called a Berliner in Berlin.

http://en.wikipedia.org/wiki/Ich_bin_ein_Berliner [wikipedia.org]

Re:Especially bland form of English, a bad thing?

[smoker2]

Big != capacious. Big = large. Capacious = plenty of room inside. Capacious, capacity. the clue's in the word itself. This is where you reductionists come unstuck. You make the mistake of assuming that words are wastefully duplicated, when usually each has a quite specific meaning, which conveys more than the simple generic term. Why struggle to make a generic term fit a situation by using adverbs and adjectives when an alternative, highly specific word already exists ? Just because you can't be bothered ?

An elephant is big, but it's not capacious, unless you hollow it out, and then it's not really an elephant anymore.