Alibaba Claims New Record in AI Language Understanding

An AI program developed by Alibaba has notched up a record-high score on a reading comprehension test. The result shows how machines are steadily improving at handling text and speech. From a report: The new record was set using the Microsoft Machine Reading Comprehension (MS MARCO) data set, which uses real questions that Bing users have asked in the past. The AI program had to read many web pages of information to be able to answer questions such as "What is a corporation?" (In this case the answer would be: "A corporation is a company or group of people authorized to act as a single entity and recognized as such in law.") Its scores were close to or slightly better than humans', according to two measures. AI algorithms have been improving at these sorts of question-and-answer tasks thanks to large, flexible learning algorithms and copious amounts of data. The Alibaba team developed a technique that essentially prunes out irrelevant text before trying to answer a question. The new program is not, however, "better at reading comprehension than humans." It was simply able to answer some questions about a subset of text better than people, on average. It is still essentially doing statistical pattern recognition without comprehending the meaning of the words it sees.

Fundamental certainty

[Anonymous Coward]

It doesn't 'comprehend', because it can't. That is outside of the realm of possibility for software, and yes, it always will be. It's baked in. I am flabbergasted that any otherwise intelligent person would think anything of software rapidly identifying flags other than, 'Wow, that was fast.'. That's all it's doing. It is doing what it was programmed to do very efficiently. This is a bad joke that has gone on far too long.

Try this?

[goombah99]

Ask Alibaba's software agent "how do I get a refund?". The ultimate comprehension test is if it finds a way to not give you a refund.

Re:

[bugs2squash]

Why is "statistical pattern recognition" different from "comprehending the meaning of the words" ?

At the end of the day if it can take a bunch of words and associate them with a sinlge enumerated concept then it has understood them

Re:

[110010001000]

It is misleading. It implies there is some intelligence behind it. It is just a computer program.

Re:

[110010001000]

Brains are not computers. Next.

Re:

[Artem S. Tashkinov]

I'm not sure you can prove it and modern physics/science says there's nothing about the brain which cannot be emulated in silicone. If I understand correctly the major difference between the software that we create and the brain is that the brain is a buttom-up design, i.e. complex processes emerge as a result of billions of neurons working together and the software is a top-down design, i.e. algorithms drive billions of transistors. As a result the brain is a lot more resilient system.

You really wanna wa

Re:

[nonBORG]

Actually modern physics/sciences says no such thing or science (modern is not a useful word here) says many things without providing proof or conclusions. Programmers know that a brain is not a computer, for the fact that they the programmer is required at all.

Not AI

[Artem S. Tashkinov]

Yet another example of something which is not AI but still being touted as AI. The test might involve comprehension but this new "AI" is still an algorithm and it neither understands the meaning of the words, nor it understands the relationship between them. It's a glorified language/data classificator.

Angel investors must be happy though.

Re:

[110010001000]

Basically every program is AI at this point. If Eliza had come out in 2018 it would be worth billions.

Re:

[pz]

You say tomaeto, I say tomahto.

More seriously, this is exactly the sort of program that will pass a Turing test.

Re:

[pauljlucas]

This [wikipedia.org] is the rebuttal you're looking for.

Re:

[dromgodis]

In my view, that thought experiment is broken from the outset: It is the room that should be considered for whether it has an understanding, but Searle looks at whether the person doing the mechanical work inside it is. Few would say that a single brain cell has an understanding.

Re:

[pauljlucas]

Eh... you could argue for it lots of ways. The room is just to ensure isolation and opaqueness from the outside. You could remove the four walls and just have him sitting at a desk in the open and it would be the same. Or the room and the stuff in it is the computer hardware and the guy becomes the CPU specifically. But he's asking whether the system as a whole has understanding.

Re:

[Livius]

We don't know what natural intelligence is. Maybe it's the same kind of comprehension only a few orders of magnitude better.

Re:

[SoftwareArtist]

Sigh. This comes up so often, I keep a document handy on my desktop with the following canned response. Please don't make claims about the meanings of technical terms if you don't know what those terms mean.

The term "artificial intelligence" was coined in 1955 [stanford.edu] by John McCarthy, Marvin Minsky, Nathaniel Rochester, and Claude Shannon (all major figures in the development of computer science). Here's how they defined it:

For the present purpose the artificial intelligence problem is taken to be that of making a machine behave in ways that would be called intelligent if a human were so behaving.

This is how the field has consistently defined it for over 60 years. Notice all the thi

Re:

[ljw1004]

Yet another example of something which is not AI but still being touted as AI. The test might involve comprehension but this new "AI" is still an algorithm and it neither understands the meaning of the words, nor it understands the relationship between them. It's a glorified language/data classificator.

You don't understand what an "algorithm" is. This is a machine-learning model, not an algorithm. An algorithm is a step-by-step series of instructions that are followed. You write your algorithm in some computer language and check it into source code repository, and you can step-by-step walk through and debug and see what it's doing. By contrast, a machine-learning model is black box made up of millions of floating point numbers that multiples an input matrix by in parallel, and combines into a numerical an

I don't understand

[nospam007]

I guess I'm not artificial enough.

Open Sesame!

[nospam007]

Did it understand that?

Oh yeah?

[the_skywise]

Ask it about the protests at Tiananmen Square.
"No protests have ever occurred at Tiananmen Square, that's just fake news."
(shades of the original Roller Ball)

Re:

[ozmartian]

hello China! *waves* at least the event/massacre, whoever you wish to describe it, is noted in Western media rather than totally suppressed... i lived in Shanghai for 8+ years and most, if not all, my work colleagues and friends there had only heard whispers about something having happened at the time but knew nothing more about it... nor did they even care to look it up which they could do as we had office VPN to bypass standard public censorship in order to do our business with the outside world....

Re:

[ozmartian]

i never defended the US, heck i agree with most of your points there!... but w/ Tiananmen Square it was "the people's" own government killing its own for protesting and then erasing it from its history... thats quite indefensible.. every year, during "the massacre's" anniversary you get censorship turned up to 1000 times its usual force online and everywhere else.. shouldn't the citizens have this knowledge? foreign wars are another evil of course but not prepetuated on citizens at home

Corporation, (Noun)

[Ryanrule]

An ingenious device for obtaining individual profit without individual responsibility. - Ambrose Bierce