Machines Almost Pass Mass Turing Test

dewilso4 writes "Of the five computer finalists at this year's Loebner prize Turing Test, at least three managed to fool humans into thinking they were human conversationalists. Ready to speak about subjects ranging from Eminem to Slaughterhouse Five and everything in between, these machines are showing they we're merely a clock cycle away from true AI. '... I was fooled. I mistook Eugene for a real human being. In fact, and perhaps this is worse, he was so convincing that I assumed that the human being with whom I was simultaneously conversing was a computer.' Another of the entrants, Jabberwacky, can apparently even woo the ladies: 'Some of its conversational partners confide in it every day; one conversation, with a teenaged girl, lasted 11 hours.' The winning submission this year, Elbot, fooled 25% of judges into thinking he was human. The threshold for the $100K prize is 30%. Maybe next year ..."

Figures

'Some of its conversational partners confide in it every day; one conversation, with a teenaged girl, lasted 11 hours.'

That's not fair, she was feeling vulnerable as she had just broken up with her N'Sync wallposter--which she had been romantically involved with for several deep & very meaningful years. Things fell apart after she saw Tropic Thunder and came to the harsh realization that an astonishing percentage [wikimedia.org] of N'Sync is homosexual.

Those soulless bots were simply preying on her emotions as they coldly recited word for word the Wikipedia entry on the band over and over.

Re:Figures

That's not fair, she was feeling vulnerable as she had just broken up with her N'Sync wallposter

Do you have to periodically replace the onion on your belt?

Re:Figures

That's not fair, she was feeling vulnerable as she had just broken up with her N'Sync wallposter

Do you have to periodically replace the onion on your belt?

Yes, yes I do [youtube.com]. What's in style these days? Vidalias? Although I'm thinking about going with leeks because--let's face it--retro is so in ...

How the conversation went with the girl:

Girl: I'm like soo depressed! He's like leaving me.

Computer: For sure. Like, ya know, like, it's so bad.

Girl: You got that straight! Like, why, like, he, like, nevar talked to me!

computer: Like, oh - my - god! Like, I like know!

Girl: Like, you know me like so good!. Like, how like old R U?

Computer: No that like older than like you.

Repeat all the above.

Yes but

Can I get it to fill in Yahoo! Buzz's Captcha for me? I've given up trying.

30% is no Turing test.

For a real Turing test, the computer must be declared human as often as humans are, and declared a computer as often as computers are.

Re:30% is no Turing test.

Has anyone done a similar test except with all humans? I'd be curious what the ratio is then. That's the number a computer would have to beat.

Re:30% is no Turing test.

You are exactly right, and that is why I think Tubleweed's comment is going to be the funniest in this thread.

The idea that humans, any human, is a fine example of perfection for AI researchers to aim for is like saying that ANY OS is a fine example of perfection to aim for. Simply because we don't abandon or throw away non-perfect humans as a rule does not mean that all are intelligent, or worthy of copying.

Apparently Geeks Should.....

study Jabberwocky's code in order to learn the logic patterns used to 'woo the ladies'.

Re:Apparently Geeks Should.....

I think this is pretty impactful. All jokes aside, the fact that Jabberwacky held an 11 hour conversation with a teenage girl is pretty astonishing. Obviuosly, a conversation of that nature is going to be all about emotion - not logic, reason or an empirical display of intelligence. Isn't that the point for AI to seamlessly interface with us? (I realize it's not necessarily the scope of the Turing test). Humans are teriible at logic and reason. Emotion is one of the key components which defines us as a species. I know a lot of humans who couldn't carry on an 11 hour conversation which primarily focussed on emotion... let alone with a teenage girl discussing nothing but fluff, pop-culture, or black and white ideologies.

I actually think it's funny, interesting and astonishing at the same time!

Oh yeah... I, for one, welcome our new teenage girl conversationalist... never mind...

This is why...

Another of the entrants, Jabberwacky can apparently even woo the ladies: 'Some of its conversational partners confide in it every day; one conversation, with a teenaged girl, lasted 11 hours.'

You need to be careful who you meet online. The rise of the machines won't be in the form of war machine. No, far worse! They're just going to woo and steal our women!

On second thought, maybe that wouldn't be so bad. That would just put everyone on equal footing with the Slashdot crowd.

The other half of the conversation

You: I was fooled.
Eliza: Why do you tell me you were fooled just now?
You: I mistook Eugene for a real human being.
Eliza: Can you elaborate on that?
You: In fact, and perhaps this is worse, he was so convincing that I assumed that the human being with whom I was simultaneously conversing was a computer.
Eliza: Why do you mention computers?

Talking to teenage girls

Another of the entrants, Jabberwacky can apparently even woo the ladies: 'Some of its conversational partners confide in it every day; one conversation, with a teenaged girl, lasted 11 hours.'

Apparently they spoke of sweet nothings.

I also welcome our new overlords to be featured in new episodes of MSNBC's "To Catch a Predator."

Easy Ways to Fool Them?

Wuoldt'n yuo tihnk a sereis of smiple tpyos that a hmn cuold undrstnd wuold fool thm? (Note that Firefox returned the first correct spelling for all but three of those words on spell check ... so maybe that's not a good example)

Or, you know, thinking up some open space game to play that is well known like truth or dare, alphabet games, association games, etc?

Or asking them open ended questions or asking them to describe love, hate--emotions that are not dictionary/wiki friendly? One would think that continually prying for personal experiences would reveal a flaw. Or perhaps simple things like "when were you born?" Followed by "how did you feel when JFK was assassinated?" if they weren't born before 1963.

I would think it quite hard to be duped into believing a program is a human.

Test the testers?

Were the testers pre-screened? Maybe the test is really showing that 25% of the population is just dumb.

Re:Test the testers?

Oh yeah? And which half am I in? ;)

Coming soon... online chat-spam-bots

This is really great news. We already have IRC bots that can fool the casual observer into thinking they are human, but this takes things to a higher level. If the source for one of these bots is available, within a few months you can expect instant messaging networks to be full of bots which are programmed to make friends with you and then after a few weeks start making subtle references to Viagra and online pharmacies. Indeed, if one of them is able to chat up the ladies, then the lonely nerd could easily automate much of the tedious work of setting up dates: get your robot to talk to thousands of potential matches at once and alert you when it gets hold of a phone number, together with a brief summary of what you talked about, and any pictures. (Or indeed, just program it to harvest pictures.) That is, if online dating works at all, which is doubtful.

Who were the judges?

I believe it is much easier to fool an average human than a person with even some basic knowledge about AI.

Artificial Intelligence vs Natural Stupidity

The day an AI will pass the Turing Test, it will be the day humanity has become so stupid to not be able to see the differences between a person and a machine.

I'm slightly nervous about all this

I'm slightly nervous about all this.

People do not think of the ramifications.

You wait until there is nigerianMalwareEliza V1 that can simultaneously hold several thousand online conversations whilst trawling for peoples information (think: dob, mothers maiden name, first school, pets name) or finding potential scam victims.

Talking to gullible teenagers is a depressing statement on modern life - hoovering out thousands of bank accounts or persuading people to part with money is a tad more serious.

I predict that soon everybody will need to watch their online chat alot more seriously.

So, I've provided one example, how else can chat bots take over the world (or at least your wallet), what are sinister uses for this technology?

Still some way to go

From The Guardian's article:

"Let's talk about religion or politics. How is the government doing?" "I'm a protestant." Oh, really? Which denomination? "I was raised as a Protestant." Then, "Judge This very minute, I am a protestant; Go ahead?"

On the other half of the screen, a faceless music fan ("I like a lot of Radiohead, Stereophonics, Led Zep etc") admitted he or she hadn't watched either the England match or X Factor last night ("Haha, Top Gear's more my style"). It was pretty clear which one was a real person. And which one the computer.

http://www.guardian.co.uk/technology/2008/oct/13/artificialintelligenceai-computing [guardian.co.uk]

Though this is quite interesting:

The event's credibility was hardly aided by the insistence of Hugh Loebner, the prize's American sponsor, that he had no interest in the result and had only set up the competition 18 years ago to promote his firm's roll-up plastic lighted portable disco dance floors.

A clock cycle away from AI?

If our criterai for AI will be so low, here's your AI: http://www.interviewpalin.com/ [interviewpalin.com].

The political side of this site aside, the answers are just prewritten answers (by a human) mixed together randomly as a Markov chain.

Does it sound convincingly? Well, at least as convincingly as some interviews a certain VP made recently. Is it AI? Hell no, a kid could write such a generator in a day.

If the bar will be as low as to try to lead casual conversations with the "AI" and expect "quirky" answers, that doesn't mean anything at all, we need to AI for this. Hell, this is what an average conversation with a teenager is most of the time. Doesn't mean it's the best they can do.

"We're clock cycle away from AI"? Please. I want my turing test to be done over an actual instant messenger program. Let's see how your Markov chain reacts, when I send a photo and ask a dead simple question such as "describe what you see in the photo".

Fooling people is easy online. Scammers do it every day, it's not AI my friends.

Did any of them

Get the extra credit question?

http://xkcd.com/329/

Here is the link they forgot to include in the sto

http://www.chatterboxchallenge.com/ [chatterboxchallenge.com]

Although the site is using some broken CSS that causes the text to render too far off to the right side, at leas in FF3

That's not possible.

Cylon leaders have three brains, and even their soldiers have one.