Follow Slashdot stories on Twitter


Forgot your password?
AI Technology

System Recognizes Emotions In People's Voices 127

cylonlover writes "Automated telephone services may get slightly less annoying thanks to research being carried out at Spain's Universidad Carlos III de Madrid and Universidad de Granada. A team of scientists from those institutions has created a computer system that is able to recognize the emotional state of a person speaking to it, so that it can alter its behavior to make things less stressful."
This discussion has been archived. No new comments can be posted.

System Recognizes Emotions In People's Voices

Comments Filter:
  • Trolling (Score:4, Informative)

    by Anonymous Coward on Wednesday November 23, 2011 @09:56PM (#38154786)
    That could become the ultimate trolling device!
  • by Anonymous Coward on Wednesday November 23, 2011 @09:59PM (#38154798)

    Why not start out with the less stressful option, just saying...

    • by Anonymous Coward

      Given the widespread availability of the Internet, the telephone is really quite a useless device now, especially for business purposes.

      When dealing with any business or institution, it is often much easier and much more effective to use a web app of some sort. These kind of apps work perfectly fine for most people, and they avoid a lot of the confusion and annoyance that can happen when having to use a telephone-based system. In the remote chance that the user needs to deal with a live person in realtime,

      • by kelemvor4 ( 1980226 ) on Wednesday November 23, 2011 @10:51PM (#38155052)

        Given the widespread availability of the Internet, the telephone is really quite a useless device now, especially for business purposes.

        When dealing with any business or institution, it is often much easier and much more effective to use a web app of some sort. These kind of apps work perfectly fine for most people, and they avoid a lot of the confusion and annoyance that can happen when having to use a telephone-based system. In the remote chance that the user needs to deal with a live person in realtime, there are many chat systems available that work just fine.

        Spoken communication is a relic of the past. It is usually no faster than using a web app or other written communication, and most of the time is much slower and much more prone to inaccuracies creeping in. In the vast majority of cases, the phone shouldn't even be an option. If it is, it should merely be the last resort.

        If that were accurate, those giant call centers that companies employ in addition to their websites would go largely unused. Turns out, lots of folks prefer to talk to someone. Just because you don't does not mean that it's a "relic of the past". While I like to do business on the web, I will most definitely avoid shopping with a company if I can't find telephone contact information easily. Web stuff works great when things are going smoothly, but typically the moment you have a hiccup in the standard process, you need to get someone on the phone to fix it. IVRU's are just a way to waste your time while waiting to be helped, so I see this effort as a mostly empty gesture. As GP said, why not start out with the less stressful option in the first place.

        • by Anonymous Coward on Wednesday November 23, 2011 @11:42PM (#38155264)
          I intentionally do business with companies that don't employ long, painful, circular IVR. I simply won't do business with a site that doesn't have a phone number.

          As you said, the web is great if you're trying to buy something. Beyond that, web interaction is mostly garbage. Many companies don't provide direct email support anymore. If they do, it usually takes 3 days to hear back because email is a nice... leisurely... game... of... ping... pong... that... they... answer... when... it's... convenient... for... them.

          Those horrible IVR's are used to deflect you back to the web. Not because their web resources are better than their people, but because humans on the phone are an expensive and time consuming resource they don't want you to use.

          So it really is a place to separate yourself from the rest. If I call your number and actually get a person that can fix problems, you're head and shoulders above your competition.
      • by mug funky ( 910186 ) on Wednesday November 23, 2011 @11:15PM (#38155164)

        yes, because all official websites are well designed, and provide many options outside the normal list that can tailor to the needs of every single user.

        sometimes a second brain is needed in a transaction.

        • Or another brain is wasted explaining something for the fifteenth goddamn time. What we need is an IVR that can detect intelligence, to screen calls, and send the idiot callers to have their explanations performed by the lowest wage brains. "If you haven't been listening to options 1-9, press 10". Seriously, dignity of human interaction is sometimes overrated when it involves angry callers insisting on the irrational, at least from the service providers perspective.
        • sometimes a second brain is needed in a transaction.

          Sometimes just the one would be nice.

    • by leaen ( 987954 )
      We should write speech recognition software to deal with those IVR
  • Or not... (Score:5, Funny)

    by NoobixCube ( 1133473 ) on Wednesday November 23, 2011 @09:59PM (#38154802) Journal

    Caller: Oh it's another damn machine.
    IVR: Do not take that tone, please, sir or madam.
    Caller: WHAT tone?
    IVR: Please remain calm, and speak clearly.
    Caller: I AM CALM, DAMN IT!!!

  • Please listen to the following staticy Muzak while you calm down. Your call is important to us!

  • even easier (Score:5, Insightful)

    by erase ( 3048 ) on Wednesday November 23, 2011 @10:06PM (#38154838) Homepage

    just have the system scan for curse words. easier than detecting emotion, and probably more accurate to determine when people are hating the IVR.

    • For some people. On the other hand, my brother-in-law uses the word 'fucking' the same way most people use the word 'um'.

    • Re:even easier (Score:5, Interesting)

      by dissy ( 172727 ) on Thursday November 24, 2011 @05:29AM (#38156396)

      just have the system scan for curse words. easier than detecting emotion, and probably more accurate to determine when people are hating the IVR.

      You would be shocked at the number of IVR systems that currently do this.
      "Fuck" is the new "mash zero"

      At least the ones I am forced to use on a regular basis most seem to support swear words as valid options. Or at least "fuck" and "shit", haven't tried any others.
      On the non-regular calls I only try this trick when my intent is to trigger "Press zero to speak with a representative", seems to be roughly 50% success rate in my semi-limited experiences.

      The only system I can think of that did not support it was my last bank. However their system couldn't be interrupted with menu presses, so you had to wait until it was finished speaking or it would ignore touch tones. Also hitting zero did not take you to a human, but back to the main menu to start over.
      I'm pretty sure they purposely designed it to make you scream "fuck!" a lot and that was listed in the design requirements specified by the customer :P

      In closing, hey Huntington if you're listening, you suck balls.
      For everyone else, give it a try next you need to press zero anyway. If nothing else, it's pretty amusing.

  • Dave... (Score:4, Funny)

    by headkase ( 533448 ) on Wednesday November 23, 2011 @10:06PM (#38154846)
    Source []: HAL: Look Dave, I can see you're really upset about this. I honestly think you ought to sit down calmly, take a stress pill, and think things over.
  • by Macgrrl ( 762836 ) on Wednesday November 23, 2011 @10:07PM (#38154848)

    ... from the Sirius Cybernetics Corporation.

  • by Zero__Kelvin ( 151819 ) on Wednesday November 23, 2011 @10:10PM (#38154864) Homepage
    If (person_calling)
    • If (person_calling)

      I usually hit 0 repeatedly until I get forwarded to a human. It's surprisingly effective.
      Staying silent also works more often than not.

      • by markdavis ( 642305 ) on Wednesday November 23, 2011 @10:31PM (#38154950)

        Not on many of the systems I encounter. It is so typical to have a poorly designed menu that does NOT offer choices that match why I am calling. When no choice is valid and I resort to "0":

        "Sorry, that is not a valid option"

        And then it repeats the 30 seconds worth of choices yet again. I have also had systems hang up on me after not answering the way they want.

        And of course the wonderful systems that ask for all kinds of information and then you FINALLY get a human and they ask for all the same damn information AGAIN.

        • by sconeu ( 64226 )

          And of course the wonderful systems that ask for all kinds of information and then you FINALLY get a human and they ask for all the same damn information AGAIN.

          Yeah, I've tried saying that I just entered the info, and they say, "sorry".

          There's a special place in Hell reserved for the guy who invented IVR menus.

          • by Vetala ( 1543063 )

            I used to work for a call center that outsourced to a mobile phone company that also had its own in-house representatives. Apparently (from what I heard) the IVR and the company's own computers were connected so that the in-house representatives actually had user accounts automatically loaded for them. We outsourced lackeys were not connected to that system however, so we got to ask the customers for all the information all over again.

        • by tisepti ( 1488837 ) on Wednesday November 23, 2011 @11:10PM (#38155146)

          Try It doesn't always have the company you need to call listed but has helped me deal with companies that ... forget to make sure yo have an easy way to get though the menu.

          • by Anonymous Coward

            Thank you so much!

        • Just press the button that's for buying something and a real person will appear almost instantaneously.
          • by wdef ( 1050680 )

            Just press the button that's for buying something and a real person will appear almost instantaneously.

            And the salesperson will throw you back onto the maddening automated menu as they (1) put you on hold to enjoy the soul destroying muzac for 30 mins; then (2) helpfully re-direct you back to the place that either didn't exit or didn't respond in the first place.

            These systems only have one aim: to stop you talking to people.

        • by wdef ( 1050680 )
          Please mod parent up.
      • Effective, but annoying. That is why my algorithm is so awesome. It removes said annoyance completely ;-)
      • by mcavic ( 2007672 )
        My favorite was the IVR at a certain cable ISP:

        IVR: "First, let's see if we can diagnose your Internet connection".
        Me: "Representative".
        IVR: "Okay, I can definitely get someone on the line for you, but first..."
        Me: "Representative".
        IVR: (in a slightly defeated tone) "Alright, please hold while I get someone on the line for you".
        • Re: (Score:3, Informative)

          by Anonymous Coward

          Sounds like Charter. I made the mistake of following their troubleshooting robot because I figured that the tech would be aware that I had done the necessary. When I got on the phone with a live human, they proceeded to run me through the exact same script as the robot so it was a complete waste of 20 minutes.

    • by Paul Slocum ( 598127 ) on Thursday November 24, 2011 @12:22AM (#38155370) Homepage Journal
      I think it's more like:

      if( person_calling_wants_to_pay_more )
      connect_to( actual_human );
      connect_to( cheap_automated_call_system )
      • by migla ( 1099771 )

        Absolutely! Not in cases where it can not apply, of course, but anywhere where there is a commercial interest and any way to use this tech to manipulate the customer, they'll do whatever they can to do just that.

      • by AmiMoJo ( 196126 )

        of even better:

        connect_to( web_site_with_same_functions )

        There should be no need to use a phone in 95% of cases, but for some reason many companies seem desperate to make everyone call them. Maybe they are lonely and just need some human contact... Oh, wait...

    • Call the sales number. Sales lines are always well staffed.
  • by Ralph Spoilsport ( 673134 ) on Wednesday November 23, 2011 @10:10PM (#38154866) Journal
    have a competent human answer the fucking phone.
    • Re: (Score:1, Insightful)

      by Anonymous Coward

      Couldn't have said it better. If the system can respond by making it less stressful for the caller, why not start out at the least stressful configuration to begin with? What possible motivation could they have for it not being less stressful from the start?

    • by _merlin ( 160982 )

      Trouble is, competence is not as common as you might like, and a competent person would probably get bored being a phone monkey, and move on to bigger, better things. The pay and work of answering support calls means less competent people will accumulate there.

      • they merely need to be competent enough. behind them are other competent people looking for a job. They move on, they get replaced. by other competent people. And they don't have to be geniuses - they only have to be competent enough to answer your questions.
        • by k8to ( 9046 )

          Having workd in, or near phone support for too many years, I will say that most are not competent.

          There's pretty basic skills:
          - Understanding the issue/problem the caller is calling about
          - Determining a response to that problem that is relevant
          - Ability to use english words to effectively communicate the reply

          Most people fall down pretty hard on one of those three basic abilities. Frequently people get bored and stop paying attention to the caller's issues at all, and give them an i

    • by Anonymous Coward

      They put you through to someone who is obviously intellectually challenged but who can actually help you so you calm your *ss down and walk him through a typical user help desk session.

      If you're dealing with someone who obviously is a doofus but a helpful doofus, you'd be surprised at how cooperative you get.

    • by PPH ( 736903 )

      have a competent human answer the fucking phone.

      Hello. This is Simon BOFH. How may I be of assistance?

    • by Krneki ( 1192201 )
      A competent human will never work in a call center.
  • Old news (Score:5, Interesting)

    by ( 245670 ) on Wednesday November 23, 2011 @10:16PM (#38154892)

    Is this one of those "On this day in history..." stories? Because analyzing caller voice patterns for stress has been SOP for the big boys for years. A pretty common way to get out of the voice prompts and to a person who will likely be competent enough to help you is to swear profusely at the voice recognition system. You'll then often be passed to a senior CSR who can get shit done. The catch is that they're authorized to hang up on you more quickly than a regular rep if you continue to swear once they're on the line.

    I guess the news here is that the existing technology is being used to present automated scripts tailored to the individual caller.

    • Re:Old news (Score:5, Funny)

      by blunttrauma ( 601130 ) on Wednesday November 23, 2011 @10:36PM (#38154980)
      I was trying to call Comcast once because the evidently cancelled my automatic billing and were now going to disconnect my internet service. When I called and the automated menu hell prompts started, I said "billing" The system replied "Sorry, I didn't get that" I let out an exasperated "Fuck". The system responded "I understand you want to talk to Billing, if this is correct, press 1" Pretty damn cool.
    • I'm awaiting the day that they turn these devices towards the reps themselves.

      Big brother could be monitoring every call and alerting supervisors real-time when the reps step out of line. Could make the job that much better.

      Man, I really miss working for Satan- *cough* I mean Sprint

    • Re:Old news (Score:5, Interesting)

      by anglico ( 1232406 ) on Wednesday November 23, 2011 @10:54PM (#38155070)
      I agree, the first time I used Fed Ex's menu for a sample return I got frustrated and 'growled' in frustration and it automatically connected me to someone who could understand why I didn't have an account number and what I needed. Of course the next few times I called I waited (just in case they added that option) and when I didn't get that option, I did the same thing, and luckily it worked.
    • That's it, I'm having George Carlin's "Seven Words" up on YouTube before I call any Tech Support number from now on.
    • Stress or anger detection is a common feature in voice-response systems. It can either elevate you like that or advance you faster in the queue. I think it only works reliably in english systems, so it might be detection of certain words. Maybe try to articulate curses to the system, or be angry in french and see if it works.


  • Easier solution (Score:4, Insightful)

    by Anonymous Coward on Wednesday November 23, 2011 @10:20PM (#38154906)

    How about just using the low-stress behavior ALL the time?

    • Re:Easier solution (Score:4, Insightful)

      by migla ( 1099771 ) on Thursday November 24, 2011 @03:01AM (#38155870)

      There was some similar story a while back about how technology like this could be used at 911, to assert whether the person calling was in an emergency.

      I would hate that, since I've noticed that while I'm a bit of a nervous person generally, when I've been in serious situations, like cutting myself badly and blood starting to gush out or being held at knife point, an unusual calm has descended upon me.

    • by Sky Cry ( 872584 )
      I wish that worked with my girlfriend.
  • by Sgs-Cruz ( 526085 ) on Wednesday November 23, 2011 @10:27PM (#38154936) Homepage Journal
    Why not, I don't know, just run the "don't stress out the human" program from the beginning? Why wait until they're already pissed off?
    • Re: (Score:3, Insightful)

      by ysth ( 1368415 )

      Because "don't stress out the human" might be actually really unhelpful stuff like slowing down speech, presenting fewer options at a time, or even fewer options at all.

  • by Anonymous Coward

    Every time i call Charter Communications to talk about bogus charges that have been added to my bill, it's like they know automatically how pissed off i am.
    The past couple times i hardly have to say anything other than "fuck" and they immediately transfer me over.

    "Press 0 if it's relating to a new plan. Press 1 if it's..."


    "We'll transfer you over to a representative."

  • []

    In the article, the researcher thought they'd have something done in around two years. This seems to be a different institution, but I guess it's nice that someone seems to have finally gotten it working.

  • by maeka ( 518272 )

    First the machines were better than me at math.
    Now the machines are better than me at emotions?

    Actually, take that back. The VAX I used in college was probably better at reading emotion than me.

  • Talking to the disembodied voice: Not happy. Talking to a person who can help me: Happy.
  • by Fished ( 574624 ) <> on Wednesday November 23, 2011 @10:48PM (#38155044)
    Computer: "Sears tool desk, can I HELP you?"
    Me: "Operator."
    Computer: "I'm sorry, I didn't understand that. Did you say, screwdrivers?"
    Me: "Salesman"
    Computer: "I'm sorry, I didn't understand that. Did you say, salepaper?"
    Me: "Cashier."
    Computer: "I'm sorry, I didn't understand that. Did you say, chainsaw?"
    Me: "NOW I want a chainsaw! I'm coming down there and #!*(%$!*%^(!"
    Computer: "I understood that! Calling 911."
  • by mrquagmire ( 2326560 ) on Wednesday November 23, 2011 @11:05PM (#38155126)
    Unless someone can make voice recognition actually works no amount of behavior altering is going to help. Maybe it's just me, but it doesn't seem like voice recognition software has come a whole lot farther than where it was when I first started playing with it over a decade ago.
    • by mcavic ( 2007672 )
      These days, I think it works quite well when the vocabulary is very small (i.e, phone systems). It does depend on your voice and your phone, of course. As a general dictation tool, no, voice recolonization doesn't work worth a damn,
      • Did you dictate that? I assume you meant recognition.

        Where I used to work, they were trying to get rid of the medical secretaries in favour of voice recognition for reporting the x-rays. It worked, but was incredibly slow because they were using a general dictionary. They changed over to a dedicated dictionary, and although it was still a little slow, for the majority of their reports it worked great. It was particularly good once the Radiologists got the hang of the short cuts. You can get through a LOT of

        • by mcavic ( 2007672 )
          Wow, did I really type that? Yes, recognition.
          • It's weird. I remember using Dragon Naturally Speaking many years ago, and then more recently. Despite many years of improvement in processor speed, memory size increases etc, it just doesn't seem to have got any faster. I've not got any stats to prove that, just my (somewhat dodgy) memory.

    • by hawk ( 1151 )

      Well, then.

      *You* obviously didn't buy a 4s.

      I'm just plain stunned by how it consistently get almost everything correct (even if Siri can't parse it)


      • No I didn't. But I do have to listen to a guy who sits by me at work talk to his 4S over and over while getting increasingly frustrated. And no, he doesn't have an accent.
  • by hawk ( 1151 ) <> on Wednesday November 23, 2011 @11:26PM (#38155204) Journal

    Simply by recognizing a few words, this can be improved.

    In particular "damnit" should be recognized.

    "I said" is another tipoff.

    but damnit is the biggie . . .

  • They already have the technology to help people when they get frustrated, it's called not making you jump through 10 different IVR menus to get to a live person.
  • Old news (Score:4, Interesting)

    by t00le ( 136364 ) on Thursday November 24, 2011 @12:57AM (#38155512)

    Real time speech analytics for call centers has existed for seven years. The better products came out of Israel, at least the first and second generation IP Telephony capable systems. Inflection based triggers have existed in traditional TDM systems for over twelve years, so not real sure why this specific article is so intriguing. Just about any high end inbound call center will use some form of inflection and emotion algorithmic processing, more so once you get into the arm pit of finance, collections.

    The systems I manage process over 500 million calls per month, across multiple industries and pbx vendors. The majority of the volume is processed using Cisco and Interactive Intelligence products, with an assortment of one off custom solutions. Of that 500 million calls per month, over 30% of the calls have used some form of inflection and emotional detection within the last eight years.

    *yawn* another slow news day?

  • They'll have the automated system scold you for your poor attitude and abusive language when you get frustrated with it, and it'll disconnect you. Also, in the soon-to-come full-on authoritarian police state we'll be living in, it'll automatically report you to the local police department, who will come and arrest you for your antisocial behaviour.
  • I can just see a sales-drone wired to a chair, getting peeved off and shouting at the consumer on the other side (has happened to me) and getting electrocuted for his efforts. That's now - if they use this system to monitor their sales-drones.
  • by NicknamesAreStupid ( 1040118 ) on Thursday November 24, 2011 @03:30AM (#38155956)
    They will be better than humans. They will know more, speak many languages, understand a huge range of contextual references, always be current, never act in a way that makes them seem rude, and even be charming. Most real humans will greatly prefer talking to them than to other humans. I'm not sure when, but some of us will live long enough to see it.
    • I cannot detect whether or not you are being sarcastic.
      • Today, how many people prefer watching television, browsing the web, or listening to their iPod over talking to their spouse or their kids? Now, image that the television, Internet, or iPod were REALLY personal and engaging and understood the viewer and responded thoughtfully and never hassled. If there is money to be made, then it will happen eventually. Now, that is cynical.
  • where the emotion recognition system will say the caller hinted..

    "FUCK YOU"

    because this is very sophisticated software.

  • The voice recognition systems employed on the typical automated phone service are horrendous. This quickly leads to frustration.

    "Q: Would you like to: pay your bill, check your balance...
    A: Pay my bill.
    Q: Sorry I did not get that. Would you like to: pay your bill, ...
    A: Pay my bill!
    A:Sorry, I still did not get that. Let's try something else. Please enter your 27 digit personal code followed by your social security and take a few minutes commenting on modern issues in Middle East politics.
    A: #$%^
    • by Splab ( 574204 )

      I've never encountered a system using voice recognition, what's wrong with using digits?
      For x press 1, for y press 2 etc.

    • by Jmc23 ( 2353706 )
      Or perhaps some people need to work on their enunciation? There should be a website to collect voice samples of everybody that complains about voice recognition systems that don't work. Nice data set to set loose some genetic algos.

      Traveling through the states there are several places where i can't understand what the person is saying.

      • by Uzuri ( 906298 )

        I was in theater and a lector for a number of years -- these systems STILL never understand what I'm saying. Evidently I can do Shakespeare, but "Yes", "No" and the digits 0-9 ain't happenin'. A shame, too, since there are perfectly useful little buttons on your phone that have worked for ages to navigate a menu.

        • by Jmc23 ( 2353706 )
          Really? Being in theater and a lector doesn't automatically mean you can enunciate. I've met plenty of both who couldn't (some theaters can't even utter a peep!). As well, what does shakespeare have to do with modern english pronounciation? Have you ever tried using the buttons? I haven't found a system yet that you couldn't just input stuff with the buttons for numbers, then again, i don't have to deal with US'ian systems. There's something seriously wrong with companies and people in the US.
          • by Uzuri ( 906298 )

            Unfortunately I have run across plenty of cases where the numbers weren't an option. These are cases where they want you to say words to get to the next step e.g. "Say 'sales' for Sales", not "Say '9' for Sales." The worst is my credit card processor, though, which for some reason allows using the keypad, but only gets it right half the time when you do (how's that even possible to screw up that badly). It's actually more successful when you speak your number, oddly.

            (And I should have made it clear that

  • foil. Even if the Bad Guy has access to the one voice that'll open the vault, some sort of tonal recognition could be developed to recognise if a person - even if authorised - can't open the vault if the system detects any sort of duress.

  • No matter you are looking for sneakers, or sexy high heels or boots, here our shop can satisfy your any requirements. We provide customers cheap shoes with great quality. And we focus on offering you comfortable and durable footwear. Let’s get rid of “cheap buyer takes bad meat” away, you will be surprised at our cheap shoes store.
  • Hi, Peggy, can I talk to your supervisor?

    Yes! Supervisor is genius! Transfer!












    Hello! My name Peggy, how help you?

"You can have my Unix system when you pry it from my cold, dead fingers." -- Cal Keegan