Become a fan of Slashdot on Facebook

 



Forgot your password?
typodupeerror
Communications Microsoft Technology

Gloves Translate Sign Language Into Auditory Speech 78

Posted by Unknown Lamer
from the babel-gloves dept.
Zothecula writes about some pretty cool sensor gloves. From the article: "Since beginning in 2003, the Microsoft Imagine Cup has tasked students the world over with developing technology aimed at solving real-world problems. In this, its 10th year, students were asked to build their project around a specific Millennium Development Goal ... The winners have just been announced ... [and winning] first place (and US$25,000) in the Software Design category was the Ukraine's quadSquad with their EnableTalk gloves that translate sign language into speech in real time."
This discussion has been archived. No new comments can be posted.

Gloves Translate Sign Language Into Auditory Speech

Comments Filter:
  • by Anonymous Coward
    Another $20 an hour [bls.gov] job gone...
    • by vlm (69642) on Wednesday July 11, 2012 @12:33PM (#40616997)

      Wrong translation direction, this going from signs to speech so a deaf person doesn't have to carry a txt2speech or a notepad and pen or learn to speak (yes deaf people can learn to speak, like one of my friends did, confuses the hell out of people who assume being able to speak means being able to hear)

      Another thing is one of my kids former school teachers worked her way thru school in the opposite direction translating speech to signs. The general impression I got was it was much closer to the fry cook pay level than the $20 claimed above. You can get $20 if you have deep technical knowledge and translate tech docs from english to Chinese, or if you have a security clearance and know Arabic or other ME languages, but...

      • by FatAlb3rt (533682)
        My wife contracts around $30/hr - so it's a bit more than fry cook. I think that it varies widely from region to region though.
      • being able to speak means being able to hear

        A lot of folks I've met can't or don't want to do the latter . . . especially politicians and managers . . .

      • by optimism (2183618)

        Wrong translation direction, this going from signs to speech so a deaf person doesn't have to carry a txt2speech or a notepad and pen or learn to speak

        If you read TFA, it was designed for people "with hearing and speech disabilities".

        Learning to speak is a big hurdle for many deaf people, but it is an insurmountable hurdle for those who are mute, even if they can hear.

        Anyway. If you're going to wear a computerized glove that can speak for you, it seems that a chorded keyboard would be a much better choice. Faster, more accurate, and more expressive than an ASL translator.

  • Old News... (Score:5, Funny)

    by inthealpine (1337881) on Wednesday July 11, 2012 @12:17PM (#40616753)
    Until I see a gorilla using them to talk about a secret diamond mine, I'm unimpressed.
  • This is a cool idea. But it kind of only goes half way. Now they need a Thing that can translate spoken speech into glove movements. Oh, yea, baby!
    • That can easily be accomplished using Speech-to-Text then Text-to-Sign.
    • Or just into words? Most people that understand sign language also understand written language.

    • by dkleinsc (563838) on Wednesday July 11, 2012 @12:33PM (#40617015) Homepage

      With that kind of glove, deaf people could say "Talk to the hand, because the ear's not listening!"

    • So, this device only actually allows the signer to sign letters that can be used to spell words, rather than translating full words and phrases.

      If that's all they were going for, then I already have a tried and true set of devices that allows for the two-way communication you're talking about, and it's far cheaper, to boot.

      Paper and pencil.

    • Well considering deaf people are the large majority of who needs this technology. A good portion of them are particularly good lip readers. Especially ones who are in the habit of being around non-deaf people. So them telling you what they want, is a tenfold larger challenge then them understanding your response.
  • by Anonymous Coward

    "Although the software was developed under Windows Phone 7, the team was forced to turn to the older Windows Mobile platform for their entry because Windows Phone 7 doesn’t provide developers access to the Bluetooth stack, which is how the gloves communicate wirelessly with a mobile device running the translation software."

    And that tells me all I need to know about WP7!

  • by wanderfowl (2534492) on Wednesday July 11, 2012 @12:28PM (#40616929)

    After watching the video, it seems that what they've done is create gloves which recognize the various fingerspelling signs. If somebody wants to sign "I need to withdraw money" (like, at a bank), what this allows them to do is to make the sign for "I", then "N, E, E, D", then "T,O", and so forth. Then the gloves feed that output into a TTS system. This works (because ASL users and English speakers share a writing system), but is horribly inefficient, and would be equivalent to a translation module that makes you speak every letter of the written words individually before putting the words into Spanish.

    This is fundamentally different from "translating sign language", where the gloves would recognize the (much more complex and spatially oriented) sign for "I", for "need", for "withdraw" and for "money", and then translate that into "I need to withdraw money" and speak it aloud. Adding in the fact that ASL syntax is fundamentally different than in English, it's quite a tall order. Interpreters need not fear.

    This is cool, nobody's denying that, and for some jobs, this might be great, but at the moment, I don't see it working much faster than taking out the requisite smartphone and writing down what you're trying to get across.

    • by FatAlb3rt (533682)
      I came here to say what you said. There's quite a ways to go on this Sign language is conceptual - they sign the concepts, not the literal words. To complete it, they need to add a display that will do speech to text for the other person. At that point, the interpreters might start to worry.
    • by cwgmpls (853876) on Wednesday July 11, 2012 @12:55PM (#40617291) Journal

      Mod parent up! Finger Spelling is *not* Sign Language. If all this does is translate finger spelling into synthesized speech, the same thing could be done much faster and cheaper by just typing the words on a standard smartphone device.

      This is not even cool. It is just, plain, wrong in so many ways. All of the money and hype spent making and marketing this device would reap 10X as much benefit if the same money were spent educating people about the real nature of deafness and sign language. The developers of this waste of time could start by taking a class about deafness themselves.

      The fact that Slashdot perpetuates the inaccurate headline equating finger spelling with sign language just demonstrates how ignorant we all are.

      • by IcyWolfy (514669)

        And the next question is... which finger spelling language are they translating? American? Irish? British? Australian? International (a mish-mash of English finger-spelling systems) ... every language uses a different set.

        • by cwgmpls (853876)
          They are not translating anything. They are transcribing within the same language. In the demo clip I saw, they are transcribing from English fingerspelling to English speech. There is no translation involved. You are right, even within a language, there can be regional variations. American fingerspelling differs from British. But they are transcribing, not translating.
      • by Jyms (598745)
        I am with you. We (http://www.coe.uwc.ac.za/index.php/SASL.html) have been working on this for a while. We are totally ignoring finger spelling for sign recognition and rendering.
    • Re: (Score:3, Interesting)

      by Anonymous Coward

      After watching the video, it seems that what they've done is create gloves which recognize the various fingerspelling signs. If somebody wants to sign "I need to withdraw money" (like, at a bank), what this allows them to do is to make the sign for "I", then "N, E, E, D", then "T,O", and so forth. Then the gloves feed that output into a TTS system. This works (because ASL users and English speakers share a writing system), but is horribly inefficient, and would be equivalent to a translation module that makes you speak every letter of the written words individually before putting the words into Spanish.

      This is fundamentally different from "translating sign language", where the gloves would recognize the (much more complex and spatially oriented) sign for "I", for "need", for "withdraw" and for "money", and then translate that into "I need to withdraw money" and speak it aloud. Adding in the fact that ASL syntax is fundamentally different than in English, it's quite a tall order. Interpreters need not fear.

      This is cool, nobody's denying that, and for some jobs, this might be great, but at the moment, I don't see it working much faster than taking out the requisite smartphone and writing down what you're trying to get across.

      I work with a deaf person, and believe me, they can sign incredibly fast. I reckon that he could say something simple "do you want to get lunch" fast enough with finger spelling that it'd sound completely normal speed to me. Sure, adding memory for all the words for various languages would be neat (very hard though), but for now, it's quite useful. Keep in mind that they don't always carry paper/phones with them, so sometimes the only option is to sign.

      • by h4rr4r (612664)

        Finger spelling is not how ASL normally works though.

        Think of even simple phrases like "thank you" no finger spelling involved.

        If they have to carry this device to finger spell they might as well use the phone it is connected to type out the message. If they are not carrying anything than this device would do them no good.

      • by cwgmpls (853876)
        Ask your coworker. They are signing, not finger spelling. Big, big difference. Using the money from this project to teach people about the difference would be much more beneficial to everyone.
    • You are right in every aspect, but you also have to factor in that this is was merely a student project. It serves extraordinary well as proof of concept. Now someone (perhaps MS?) needs to focus on it and improve it by making it able to translate the actual sign language gestures. If fingerspelling is possible, the jump to sign language should be not that hard.

    • where the gloves would recognize the (much more complex and spatially oriented) sign for "I"

      Actually, the sign for "I" and the sign for the letter "i" are pretty much identical... but your excessively intensified and outrageously exaggerated point is well taken.

  • With a text to speech app?

    • by Kergan (780543)

      With a text to speech app?

      Presumably because it's faster to sign than it is to type, in much the same way as it's faster to speak than it is to type.

      • by cwgmpls (853876)
        Presumably, but its not. These gloves transcribe finger spelling. Finger spelling is not a separate language, it does not require translation, and no deaf people use it as their primary form of communication. Any deaf person who knows how to type can easily type faster than they can finger spell.
      • With a text to speech app?

        Presumably because it's faster to sign than it is to type, in much the same way as it's faster to speak than it is to type.

        Indeed, it is.

        Unfortunately, the glove does not translate sign language, but rather transcribes finger spelling.

        In this case, the text-to-speech app would be superior.

  • Not going to work (Score:5, Informative)

    by Cyberax (705495) on Wednesday July 11, 2012 @12:36PM (#40617073)
    The gloves translate dactylology (finger spelling). That's fairly easy, but it's horribly inefficient.

    'Real' sign languages (like ASL) are much harder to translate because they are somewhat non-linear. A single gesture can describe several things at once: size, direction, emotional state, etc. There's no way you can translate it without fully understanding the context of the speech. And we all know how good computers are at such tasks...
  • Wow! That's new! Speech never used to be "auditory" before. That's a real breakthrough. Maybe these guys can tackle the old problems of Visual Sight, or Sensory Touch, or even ... Olfactory Smelling.
    • Olfactory Smelling.

      I am anosmic, you insensitive clod!

      (btw, I really am anosmic, a smelling device would be awesome :p)

  • A grad student at Stanford at the Center for Design Research did this in the mid-1980s. It had to be connected to a workstation back then, of course.

  • Close palm, raise middle finger. Universal language at it's best.
  • Then I guess it translates sign language. Finger-spelling is just an alphabet. I don't know about other sign languages, but ASL is a full language with its own grammar and conventions, and it would take a lot more than a glove to interpret it. Positions of hands with respect to the body are important, as are facial expressions, and ASL's pronoun system is largely spatial with the handshape only indicating the type of pronoun (e.g. personal vs. possessive). Even if a piece of technology could reliably capt

"Bureaucracy is the enemy of innovation." -- Mark Shepherd, former President and CEO of Texas Instruments

Working...