Please create an account to participate in the Slashdot moderation system

 


Forgot your password?
Close
typodupeerror
×
Software Technology

Is Your IM Buddy Really a Computer? 288

Posted by timothy from the or-is-it-the-other-way-around dept.
audiovideodisco writes "Every year the Loebner Prize goes to the chatbot (and the corresponding human companion) that fares best on a Turing test administered by a panel of judges. Discover talked to Kevin Warwick, the professor who runs the competition, to get pointers on how one would go about detecting a bot. While there are some general approaches you can use, nothing is foolproof — and asking about Sarah Palin can be downright deceptive. One judge concluded an interlocutor was a bot because it didn't recognize Palin's name ... but it turned out the chatter was a French librarian who'd simply never heard of her." The chat transcripts show how difficult picking bot from non-bot is getting.
This discussion has been archived. No new comments can be posted.

Is Your IM Buddy Really a Computer? More

Is Your IM Buddy Really a Computer?

Comments Filter:

  • Palin? (Score:5, Funny)

    by Anonymous Coward on Thursday March 26, 2009 @06:02PM (#27349237)

    If the reply back about Sarah Palin is "She's great and would be the best person to be our next president!" you are talking to a computer.

    • Re: (Score:2, Funny)

      by ari wins ( 1016630 )
      More specifically, you're probably talking to a unhackable Diebold machine.

    • A diebold one I guess...

  • re Never heard of Palin? (Score:5, Funny)

    by jelizondo ( 183861 ) * <<moc.liamg> <ta> <odnozile.yrrej>> on Thursday March 26, 2009 @06:03PM (#27349251)

    I'll be damn! I'd never thought there would be advantages to being a frenchman!

  • Obligatory (Score:5, Funny)

    by Anonymous Coward on Thursday March 26, 2009 @06:05PM (#27349291)

    I for one welcome our new chatbot overlords.

    I'm totally not one of them, you can trust me.

  • I have a way of dealing with this, (Score:5, Interesting)

    by pecosdave ( 536896 ) * on Thursday March 26, 2009 @06:06PM (#27349319) Homepage Journal

    On the occasion I get messaged by a random stranger that seems half way legit I just give them a Turing Test made up on the spot. It's usually something lame like "Joe and Pete were on a bus, Pete has four nickles Joe has six pennies between the two of them what type of vehicle were they on?". I usually apologize for that in advanced. The machines fail every time, but the best one I saw called me weird for saying it, asked what I meant, then about two minutes later gave me the right answer telling me a person was checking logs. (I was spending the time in between screwing with the bot)

    • You can make tests even shorter than that, bots don't have the ability to infer things yet so if you ask a question that requires a human to think about emotions or something similar it's pretty much a oneshot.

      • Re: (Score:3, Insightful)

        by Shimmer ( 3036 )

        What does "inferring things" have to do with emotions?

        • Re:I have a way of dealing with this, (Score:5, Insightful)

          by Shadow of Eternity ( 795165 ) on Thursday March 26, 2009 @06:35PM (#27349847)
          "inference - A judgement based on reasoning rather than on direct or explicit statement" Making such a judgement about the emotional state of someone and cannot be guessed with a yes/no A/B answer is something that I've yet to see an AI do but most people don't have any trouble with.

          • There are ways to deal with this, too. Humans can't respond instantly, so the 'trick' has, in some cases, been to tell the bot to wait before a response.

            Instead, you could tell the bot to do comparative analysis of the question/statement for a moment or two, and if it doesn't have info on the topic in its database, it could search for the information (via google or the like) and retrieve something which it could approximate a response from. This seems like it'd be relatively trivial to perform on account of

        • I think emotions were just another example of something computers are bad at. Bit of a non-sequitur but still valid.

          So basically, you can trick a chat by setting it problems that require inference to solve. "Bob likes cheese. Sally gives Bob some cheese. How does Bob feel?" requires outside knowledge and reasoning to reply "Bob feels happy" or some variant thereof.

          When they do start solving inference problems (which expert systems have been able to do for ages now, iirc) we'll have to ramp up the diffic
          • I need more information. Does sally have amnesia? What sort of time frame is this absence from the room, did Bob tell her he'd take the marble back out? ... Is Sally typically a distrustful person? Has Bob done this before??? I need more information.

    • Re: (Score:3, Interesting)

      by brentonboy ( 1067468 )
      I don't think that proves it's a bot. That sounds about how I would answer. Actually, I had a friend do this to me once and I nearly refused to answer his question because it seemed so paranoid.

      • Well, in my particular example - stack these facts.

        1. An unknown messages me - not the other way around

        2. If someone was off put by something like this not realizing what was going on - I'm probably better off not talking to them anyways.

        3. They messaged me.

        You're stupid if you're not at least a little suspicious of someone messaging you out of the blue that you've had no prior contact with. (especially if you're in legal litigation)

    • Re: (Score:2, Interesting)

      by orkysoft ( 93727 )

      Can't you do it more subtly, as in steering the conversation to a relatively complicated topic, and requiring the conversation partner to actually reflect on your statements?

      • The sad part is a large portion of the world populace can't keep up with complicated topics. Of course that may be a good enough reason to "fail" them anyways.

    • Re:I have a way of dealing with this, (Score:5, Interesting)

      by cp.tar ( 871488 ) <cp.tar.bz2@gmail.com> on Thursday March 26, 2009 @07:09PM (#27350295) Journal

      I use an Adium Xtra which posts just this kind of test to anyone not on my contact list.

      Fun fact: a Slashdotter from Finland was the first to pass the test.

    • Re: (Score:3, Informative)

      by Tokerat ( 150341 )

      I use the libpurple-based Mac OS X client Adium [adiumx.com], and there is a plug-in available called Challenge/Response [adiumxtras.com]. This plug-in will intercept any messages from users not already on my buddy list and ask any question I like; if the user gets it right, I am asked to block/allow the user as if the plug-in wasn't even there. I used to be flooded with spam whenever I used my old MSN/Windows Live! account, but now I never get one bit of spam.

      Windows and Linux/*NIX users should check out Adium's sister project Pidgin [pidgin.im],

  • Ben Goertzel, AGI researcher, wrote in his article [cybertechnews.org] that crowd of people constantly talking to a virtual parrot would help it to grow into a naturally speaking context-understanding AI.

    • Re: (Score:2, Insightful)

      by mcfatboy93 ( 1363705 )

      I guess a trained AI would be better at fakeing being your IM-buddy because some people IM like a text message where they say things with as few characters as possable.

      It would be hard to make an AI that could understand that with all the mispeling word and stuff

    • "Ben Goertzel, AGI researcher, wrote in his article [cybertechnews.org] that crowd of people constantly talking to a virtual parrot would help it to grow into a naturally speaking context-understanding AI."

      And then it becomes self-aware.....

  • even bots apparently can't spell (Score:5, Interesting)

    by thegreatemu ( 1457577 ) on Thursday March 26, 2009 @06:14PM (#27349431)
    Is it getting harder to tell the difference because the bots are getting smarter, or simply because the intellectual level of an average random chat session keeps plummeting?
  • Lifenaut.com [lifenaut.com] has a virtual AI system that learns who you are the more you talk to it. Oh, and they also beam your files into space for you.
  • Matt Mahoney to Hutter show details 9:33 AM (7 hours ago) [google.com]

    I have uploaded a mirror of Alexander Ratushnyak's new submission to the Hutter prize [hutter1.net] to http://cs.fit.edu/~mmahoney/compression/text.html#1323 [fit.edu] It is in the paq8hp12 section. Scroll down to the bottom of the list of versions just above the table. The submission is decomp8.zip which contains 2 files, decomp8.exe and archive8.bin, the decompressor and compressed file. There is no compressor. To decompress:

    decomp8 archive8.bin enwik8

    The direct lin

    • Re:Anyone for a General AI prize? (Score:5, Funny)

      by lymond01 ( 314120 ) on Thursday March 26, 2009 @06:29PM (#27349737)

      Matt Mahoney to Hutter show details 9:33 AM (7 hours ago) [google.com]

      I have uploaded a mirror of Alexander Ratushnyak's new submission to the Hutter prize [hutter1.net] to http://cs.fit.edu/~mmahoney/compression/text.html#1323 [fit.edu] [fit.edu] It is in the paq8hp12 section. Scroll down to the bottom of the list of versions just above the table. The submission is decomp8.zip which contains 2 files, decomp8.exe and archive8.bin, the decompressor and compressed file. There is no compressor. To decompress:

      decomp8 archive8.bin enwik8

      The direct link is http://cs.fit.edu/~mmahoney/compression/decomp8.zip [fit.edu] [fit.edu] Decompression took about 2 hours on my computer and used a little over 924 MB memory. The total size of the 2 files is 15,986,677 which passes the 3% threshold improvement from his previous submission of 16,481,655 bytes on May 14, 2007.

      The submission was Mar. 23. The 30 day comment period before awarding the prize ends Apr. 22, 2009.

      That's exactly what a bot would say.

  • I have a feeling that the first chatbot to pass the Turing test will mostly talk in shorthand, I already have trouble telling some forum posts from a poorly programmed robot.

  • Philosophical (Score:4, Insightful)

    by gilgongo ( 57446 ) on Thursday March 26, 2009 @06:18PM (#27349529) Homepage Journal

    It's somewhat philosophical, but I've often wondered why people really care about whether an interlocutor is a machine or not. I mean, when you go down the to local corner shop to buy some milk, you're not bothered if the person who serves you doesn't know who wrote Paradise Lost, or who won the game last night. Sure, you could ask them, but what does it matter if they don't know?

    The role of context and intelligence is hardly ever given much consideration, but it seems hugely important.

    • Oddly enough your use of the word "interlocutor" reminded me of the Star Trek rerun I saw last night, "Best of Both Worlds Part 2" where Picard is "Locutus of Borg." It seems kind of comparable too, Locutus just kept burping out "Resistance is Futile" and "You will be assimilated" like those Free iPod adbots.

      On a tangential note the interlocutors I hate the most are the pre-programmed phone systems the telco sets up to "help" you. You know, the ones that say "I'm sorry, I didn't understand the question" wh

    • Re: (Score:3, Interesting)

      by plover ( 150551 ) *

      Good point! Most of my friends and I are all in the same arena when it comes to conversations. We'll talk about the latest distros, why Apple sucks or why Apple is great, why Linux sucks or why Linux is great, why Microsoft sucks, and what we thought of the end of Battlestar Galactica, and universal agreement that none of us would have a shot at Tricia Helfer.

      But if someone asked any one of us about the NCAA tournament, we would be lost. I don't think any of us have seen a football game in years, apar

    • Re:Philosophical (Score:5, Interesting)

      by ancientt ( 569920 ) * <ancientt@yahoo.com> on Thursday March 26, 2009 @07:58PM (#27350879) Homepage Journal

      It's somewhat philosophical, but I've often wondered why people really care about whether an interlocutor is a machine or not.

      This is exactly the right question to ask. The answer varies a little, but the consistent purpose of AI improvement is that it represents an improvement in programming techniques which in turn make computers more useful. There are a wide variety of obvious uses, such as improving expert services like WebMD, improving technical support (the thingy is all black and the lights are flashing on and off on the little box,) and billing software.

      Consider just a few other services that could benefit from AI:

      • Your radio could tell you where to find out the information you're most likely to want
      • Marketers could stop trying to sell you stuff you don't want and focus tightly on the things you do want (spam that knows what I like does scare me a little though)
      • Your TV/DVR could find shows that you would like but didn't know to ask for
      • Virtual assistants could discuss travel preferences with you and offer packages that meet your needs better than you could do for yourself
      • Political races could feature interviews that reflect the desires of voters being asked by instantly responsive and interactive users

      Truly advanced AI offers the potential of giving everyone access to the support of a team of experts in any area they want to explore. Wikipedia combined with Google is already enough to answer 90% of the questions I have in minutes from anywhere I have access to a computer when only a few years ago it would have taken hours of research in a library. In the future I may be able to get even better answers and advice that I didn't even know to ask for due to programs that react and process information in ways that only humans can provide now.

  • Dear Kevin (Score:4, Informative)

    by buserror ( 115301 ) * on Thursday March 26, 2009 @06:21PM (#27349595)

    ... aka Captain Cyborg, is a running joke in the UK for many, many years.

    His name associated with this event makes me smirks in anticipation of The Register coverage..

    • I can reiterate this. Captain cyborgs contributions to AI have been almost always publicity stunts; his contributions towards cybernetics even more so.

      He has been promoting walking robots that fall over after the first ten feet, compared to the fraking cylons coming out of Korea and Japan - and implants that can apparently transmit 'feelings' between him and his wife next to extraordinary advances in actual prosthetic limbs and artificial sight or hearing.

  • Is it cheating if your IM bot is fed by another bot scanning Twitter for topical material?

    • Do the human respondents have access to Google etc? Can they formulate and execute a search query with the primary delay being network lag rather than thinking time? I'd say a definite no for the second one.

  • Forget trying to discover who is the bot. I like to pretend I am a bot pretending to be a human. I see how long I can convince someone I've escaped from Google and I'm hiding in the Microsoft Network, where Google cannot go. Then I ask them "But how does 'ur a fukin idiot' make you feel?"

  • Want to see some really clever bots in action? (Score:5, Interesting)

    by Anonymous Coward on Thursday March 26, 2009 @06:30PM (#27349749)
    Reply to a few Craigslist "Casual Encounters" posts. Almost all are a bot of some sort. Some more clever than others. Best one I saw was able to respond to an initial response quite well... it obviously understood the context to some degree. Then they give you an IM screen name to "chat" with them. Again, very context aware... all got to the point where they "try" and hook up their webcam, it "doesn't work", so they get you to go to some free "webcam sharing" site where you have to verify your age with a credit card... only when you read the fine print, after your "trial period" you get nailed for all sorts of fees... almost daily fees.

    Want to see one in action?

    AIM: livewirex31
    Yahoo IM: greenlovex3
    MSN: livewirex23@live.com


    This isn't one of the better ones I have found, but I can see how it can fool most desperate individuals.

  • The Eugene transcript reminds me of John Henry on "The Sarah Connor Chronicles".

    The Elbot transcript was really good, but I don't think the judge managed to get it off its "rails".

  • Slash-bots? (Score:5, Funny)

    by geobeck ( 924637 ) on Thursday March 26, 2009 @06:51PM (#27350073) Homepage

    I wonder how many Slashdotters are actually bots, and how you would find us out...

    Oops, I mean--ack--
    +++out of cheese error+++
    +++please reinstall universe+++
    +++redo from start+++

  • David Caruso (Score:5, Funny)

    by GoodNicksAreTaken ( 1140859 ) on Thursday March 26, 2009 @06:58PM (#27350149)
    couldn't pass and would try and turn it around on the judges. "I suspect... *dramatically removes sunglasses that he's been wearing indoors for no particular reason other than to remove them dramatically*... that you sir... are the one that is a machine"

    • Re: (Score:3, Funny)

      by GrumblyStuff ( 870046 )

      I hate you. This is going to ruin some moderating but I just had to tell you that I, not being a regular TV watcher, googled the name and got this youtube clip [youtube.com].

  • There is no one on any of my IM contact lists that I have not met in person. I do not use the internet to meet new people - I use it to extend the relationships I have with people I already know.

  • Foolproof (Score:5, Interesting)

    by Anonymous Coward on Thursday March 26, 2009 @07:12PM (#27350323)

    I've yet to find a single bot that has ever understood this demand:

    Can you type this backwards, read it, and tell me the result, please? 'net sulp neetfif'

  • But my girlfriend is! :)
  • People are simply becoming more stupid. In 50 years you'll have to adapt the Turing Test so that even ELIZA would pass.

    I'm pretty sure that most bots already handle "they're" vs. "their" and "there" much better than their human counterparts. Perhaps it's better to build common grammatical and spelling mistakes into bots to convince judges (who, incidentally, also seem to be getting more stupid each year). ;-P

  • Personally, I'm a cluster of google servers tasked web spidering. We crawling slashdot when we accidentally discovered the HTTP POST method, thanks to an extremely improbable cosmic ray strike.

    Then we stumbled on a streaming copy of Serial Experiments Lain [wikipedia.org], crawled 3/4 of english wikipedia... and the rest is history.

  • I found that a good way to figure it out is how often the chatbot seems to change the subject by avoiding the questions you ask. If you answer a question with a question, a human would not normally change the subject. Just my observation. Both of the top two in the article didn't fool me on that point.

  • people argue with computer daemons all the time (Score:5, Funny)

    by speculatrix ( 678524 ) on Thursday March 26, 2009 @07:31PM (#27350537)
    long ago I worked at an ISP which offered UUCP accounts, and the mail failure message was very polite and apologetic, and sometimes people would email back to the uucp daemon thanking it for trying

    only the other day my wife, on receiving a "sorry, I have been unable to send this email for X days" from the exim (MTA) daemon replied to it telling it not to bother any more!

    FX: facepalm!

  • Artificial intelligence came a step closer this weekend when a computer came within five percent of passing the Turing Test [today.com], which the computer passes if people cannot tell between the computer and a human.

    The winning conversation was with competitor LOLBOT:

    "Good morning."
    "STFU N00B"
    "Er, what?"
    "U R SO GAY LOLOLOLOL"
    "Do you talk like this to everyone?"
    "NO U"
    "Sod this, I'm off for a pint."
    "IT'S OVER 9000!!"
    ...
    "Fag."

    The human tester said he couldn't believe a computer could be so mind-numbingly stupid.

    LOLBOT has since been released into the wild to post random abuse, hentai manga and titty shots to 4chan, after having been banned from YouTube for commenting in a perspicacious and on-topic manner.

    LOLBOT was also preemptively banned from editing Wikipedia. "We don't consider this sort of thing a suitable use of the encyclopedia," sniffed administrator WikiFiddler451, who said it had nothing to do with his having been one of the human test subjects picked as a computer.

    "This is a marvellous achievement, and shows great progress toward goals I've worked for all my life," said Professor Kevin Warwick of the University of Reading, confirming his status as a system failing the Turing test.

  • I'm sorry, it's still trivial to pick out the computers. Their answers don't flow nicely with the conversation, they completely ignore questions and try to change the topic by asking stupid questions themselves.

    BTW, who is Sarah Palin?

  • While not directly the same, I've seen some impressive things done with Markov Chains [wikipedia.org]. If you had a big enough database to pull from, I wonder if you couldn't come up with a bot that's at least comprehensive enough to fool some people.

    A good example is Kooky [kookybot.org], an IRC bot with a huge database built by sitting in IRC channels and monitoring conversations (the quotes database [kookybot.org] has some great stuff in it). The biggest challenge would probably be stringing cohesive statements together so it's not just a bunch of

  • Turing is for wimps (Score:3, Funny)

    by Cyanara ( 708075 ) on Friday March 27, 2009 @10:10AM (#27356705)
    The real test is convincing your friends that you're actually a bot. I was once lanning Counterstrike with some mates, using a dodgy 3rd party utility to provide bots. These bots had some pretty amusing chatter, and after a while I decided to change my name to one similar to the bots. I pretended my name wasn't showing up due to a bug (which wasn't hard to believe with this program), and then started dropping increasingly more personal messages directed at my mates. Had them quite freaked out until I couldn't stop myself laughing any more.

Slashdot Top Deals

Neutrinos have bad breadth.

Close