Stories
Slash Boxes
Comments
Search

News for nerds, stuff that matters

Slashdot Log In

Log In

Create Account  |  Retrieve Password

Sign Language Via Cell Phone

Posted by kdawson on Tue Feb 13, 2007 06:09 AM
from the can-you-see-me-now? dept.
Communications Handhelds Hardware Science
QuatumCrypto writes "A project is underway at the University of Washington to enable real-time sign language communication via cell phone. Because of the low-bandwidth wireless cell phone network, a new compression scheme is necessary to capture only the bare essential components of signing to minimize data transfer. Although text messaging is a viable alternative for everyone, signing — like speech — is a much faster and more convenient form of communication."
+ -
story
This discussion has been archived. No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.

Sign Language Via Cell Phone 50 Comments More /

 Full
 Abbreviated
 Hidden
More
Loading... please wait.

  • Video calls (Score:3, Interesting)

    by dotancohen (1015143) on Tuesday February 13 2007, @06:14AM (#17995356) Homepage
    I've already seen sign language being used over video calls. Then again, as one who volunteers with autistic children, I've seen a lot of super-use of technology and hands...
    • Just out of curiosity, how does this work? Did they get someone else to hold the phone for them, or can ASL work with only one hand?

      • Re: (Score:2, Informative)

        by dotancohen (1015143)
        Yes, we hold the phone for him. I've also got a Nokia 6280 with video calls. The video calls at 0.46 NIS/m are cheaper than regular phone calls at 0.67 NIS per minute. The boy is question is not only autistic but also mostly deaf. What's interesting is watching young kids talk on the phone. Even on a non-video call they nod yes and no.

    • I've seen a lot of super-use of technology and hands...
      ... and we're all very familar with that around here.

  • They're focusing on video... (Score:5, Insightful)

    by Mr2001 (90979) on Tuesday February 13 2007, @06:16AM (#17995368) Homepage Journal
    But there's another problem with using sign language via cell phone. Look at the screen mock-up on that page - it shows the signers from the waist up. If your phone is far enough away that it can capture your whole body, how are you going to see the screen?

    Also, they claim "The current wireless telephone network has inadvertently excluded over one million deaf or hard of hearing Americans", but it's easy to get a cell phone that supports TDD [phonescoop.com], just like a wired phone.

    • Re: (Score:2, Interesting)

      by accessbob (962147)
      I was at a presentation of their paper on this in Portland last year: http://portal.acm.org/citation.cfm?id=1169001&jmp= cit&coll=ACM&dl=ACM&CFID=14265233&CFTOKEN=82641255 #CIT [acm.org] From the abstract: "...techniques that exploit the visual nature of sign language. Inspired by eyetracking results that show high resolution foveal vision is maintained around the face, we studied region-of-interest encodings (where the face is encoded at higher quality) as well as reduced frame rates (where few

    • it's easy to get a cell phone that supports TDD

      That's all well and good, but that requires carrying around a TDD keyboard in addition to the cellphone. Those things aren't small. It also requires that the receiving party also have a TDD, unless the cellphones know to display the TDD text on their tiny screens.

  • How do you hold the cell phone? (Score:4, Insightful)

    by niconorsk (787297) on Tuesday February 13 2007, @06:17AM (#17995372)
    The technology for this is very cool and all, but I don't see it as very applicable to use with cell-phones. As far as I know requires the use of both hands, so you would have to put down your phone in a way that you can be seen and you can see the screen and lastly without holding it. This seems like an impossible proposition. But the technology in its own right could be very interesting, at least for desktop video-conferencing units.
    • I know requires the use of both hands, so you would have to put down your phone in a way that you can be seen and you can see the screen and lastly without holding it

      That was my first thought as well, but it will be nice to have the software ready for when little wireless spec displays & cameras are available (if they haven't thought of a solution already).

  • Language-agnostic? (Score:3, Insightful)

    by Max Romantschuk (132276) <max@romantschuk.fi> on Tuesday February 13 2007, @06:17AM (#17995374) Homepage
    I hope this compression scheme won't be tied to the semantics of a single sign language like ASL. There are plenty of other sign languages in the world, so hopefully this tech will be "language-agnostic", so to speak.
      • They're fairly simple solutions for face and hand tracking and work well under the assumption that everybody has the same color of skin, but we all know this isn't really the case.

        Well, there's a pretty simple solution to this problem. We just need to pass some legislation stating that you can't be deaf unless you have a certain skin tone.

  • Makes sense (Score:4, Funny)

    by $RANDOMLUSER (804576) on Tuesday February 13 2007, @06:23AM (#17995408)
    I often use sign language to people using cell phones while they're driving.
  • I still think deaf people should communicate by getting to kick non-deaf people in the crouch. It works similiar to morse code, but with "crunches" and "squishes" instead of "dots" and "lines".

    But I'm one for giving handicapped people excuses to hurt the rest of us. It just seems fair. And I wear a cup.

  • no subject (Score:5, Informative)

    by UnixSphere (820423) on Tuesday February 13 2007, @06:53AM (#17995546)
    "What is the benefit over txt messaging?"

    Sign language is much faster obviously, and sign language is based alot on the user's emotions and how they use a certain sign or signs.

    But to answer the parent's question, none of the cell phone carriers offer a price break for deaf/hard of hearing users.

    BUT the deaf community is fond of using the t-mobile sidekick, all versions, because of the relatively cheap unlimited txt/data plan that comes with it. Sidekicks are almost dominant among deaf people. Some deaf tech sites and companies offer the sidekicks significantly cheaper to deaf users since it is so popular among them.

    • A while back, I had a neighbor who was deaf. I helped him and his hearing wife with their computer a few times for free. (And not out of pity, because I didn't know he was deaf when his wife asked for the help.)

      Anyhow, he also used a sidekick. Unfortunately, I know this because I found it in the parking lot, run over.

      He was a nice guy, but a little too eager to communicate with other people. He came across as simple because of it, but I don't think he really was.

      • Don't some morse characters have 5 or 6 dots or dashes? what's the average length of a character in morse? I would have thought that the current phone keyboards where most characters are between 1 and 4 characters away would be faster. Plus the cognitive leap of having to learn another intermediary language... ? All numbers on a current keyboard are one key press, how does this compare to morse?
  • No, what you need is a pair of wiimote-like gloves that you wear which are connected to a tiny robot monkey on the recipients cellphone that mimics your movements. The recipient, in turn, wears another pair of gloves which are connected to the robot monkey on your phone.

    OK, so instead of a robot monkey you could have a little animated monkey on your display, but a robot monkey would be better. Tiny robot monkeys is how Apple will implement it on the iPhone while the rest of the industry just has animated
  • Although text messaging is a viable alternative for everyone, signing -- like speech -- is a much faster and more convenient form of communication

    Speech is flavored in languages, like text. So speech is not convenient at all if this is what they are saying. Otherwise, signing is not more convenient because only a small fraction of people already know it. I'm confused. Someone explain it to me.
  • They are flocking to the Sorenson VP-100 system.

    I cannot, for the life of me understand this, when there
    are so many video based chat sites on the net.

    All the deaf people I know have PC's. I met my first
    deaf friend on the old BBS's. In the text messages on
    FIDOnet.

    I would not want a deaf user signing while driving :P
    • I would not want a deaf user signing while driving :P

      It's like I learnt in Italy - you NEVER speak to an Italian when he/she is driving, because they are forced to take both hands off the wheel to reply to you!

    • I would not want a deaf user signing while driving :P

      That might not be so bad... The ability to hear and use one's hands for driving don't seem to do much good for the vast majority of the public. At least deaf people would be are used to it.

  • Just hack Wiimote! (Score:3, Insightful)

    by 140Mandak262Jamuna (970587) on Tuesday February 13 2007, @07:30AM (#17995742) Journal
    Another big chance for Nintindo. Can they hack the Wiimote to translate sign language to text?
    • Sign language is more in the fingers, the wii-mote would be useless here.

      Stop modding him insightful.
      • Well, the Wii can handle four wiimotes simultaneously. You need something like ten motion sensors to do sign language. But sign language to text wiimote will be covered by medical insurance as a "medically neccessary prosthetic device". True, gamers are willing to pay insane price for the "in" thing. But the profit margins in medical devices is an order of magniture higher than game consoles. For example the bluetooth ear pieces are being thrown in as freebies or being sold at scrap yard prices. But the hea
        • Now that's an insightful post. I'm sorry I doubted you, but you really spoke like you didn't know what you were on about before. I still don't think that the wiimote is a good basis for a sign-to-texting device (making one at all is still a great idea), but using a wii for physio seems like a good idea.

          There have been therapy-based consoles before, I seem to remember one for treating something like ADD that paused the game whenever the player stopped paying proper attention. If they improved the wiimote's s
    • I think you're joking. But just in case, in BSL at least, sign language relies on nuance to form different words - like lip shapes, facial expressions, etc.. For example you can do a sign for "lemonade" and for "to f***" (in the sexual sense, not in the fsck sense) that differ only in the facial expression.

      Deaf kid (signing): Mom, don't forget to buy lemonade for Dad

      Mom: Wait till I get home you dirty little brat!

  • Videophones (Score:4, Informative)

    by Zarhan (415465) on Tuesday February 13 2007, @07:44AM (#17995822)
    At least in Finland, in cooperation with a Finnish hearing-impaired association, there's been some projects with 3G video-phones. Yes - selling a phone to deaf people opens up a nice new market :). Anyway, as far as I know the experiences have been overall positive - and no fancy sign-language-specific codecs or anything, just a normal 64kbps video phone call and a camera phone.

  • Sign language over mobile works on 3G already (Score:4, Informative)

    by threepoyke (1063604) on Tuesday February 13 2007, @08:08AM (#17995944)
    There is nothing new about this story. Sign language over mobile (cell) networks already works with regular 3G (UMTS) phones in Europe. Take a trip to Örebro in Sweden, which has a high concentration of hearing impaired due to a specialist education cent(e)r(e), and you'll see loads of teenagers using their 3G phones to talk using sign language. In the streets, on the bus, in cafes, everywhere. This article http://svt.se/svt/jsp/Crosslink.jsp?d=37482&a=5369 32 [svt.se] (in Swedish) from February 2006 even talks of the local social security services offering customer service to hearing impaired using 3G phones and sign language.
  • There's somthing very stupid with the concept of a sign language chat via cell phone. People are limited by the capabilities of a phone, and at the same time, given the capability, people will use it as they please. That means that no such 'sign language phone' exists (or will ever exist).

    Either a phone has video transmitting capabilities, or it doesn't. If it doesn't, then there's no hope of having a sign language chat (unless we use CGI to simulate it, which would be fancy texting). If it does, then optim
  • Siging is good for the deaf, but the blind have trouble using cell phone because they all react to input graphically instead of audibly.

    • Sign language text and language (Score:3, Informative)

      by Anonymous Coward
      Not all deaf can just use text. Some know only sign and don't know any English. It is impossible for them to use TTY's or other text. And even if it is possible, it can be slow and painful. Learning English text when you can't hear is a very very difficult thing to master. Especially literacy in a second language when you're illiterate in your first language. Stefan Wöhrmann in Germany has had great success teaching German text to deaf, but he uses written sign language for teaching.
      • Sorry what?

        If they know sign, can't you sign to teach them to read? I can't imagine someone being very functional in society without any written language knowledge.

        Tom

        • If they know sign, can't you sign to teach them to read? I can't imagine someone being very functional in society without any written language knowledge.

          First important thing to know: ASL isn't signed english - it's a language very much of its own. And for those who are born deaf it's their primary language. Most hearing people can learn a secondary language but with very different degrees of success and I guess it's the same for the deaf.

          Consider the following thought experiment: imagine you growing up

          • I know that ASL is it's own language. I'm just saying if you have a deaf kid, you'd immerse than in English [or whatever the standard is] written text as much as possible as early as possible.

            I wouldn't expect them to SPEAK it easily, but reading shouldn't be that hard.

            Nobody "speaks" C, yet i can express ideas in that language easily.

            Tom

            • I think you're missing the point about ASL being a completely different language. It, unlike most spoken languages, does not have a written component. For someone who grows up in a completely deaf family and culture, it's more than just a language barrier.

              You had the benefit of growing up with a language that was easily expressed in two ways, something those with ASL as a first language do not always have.

              • ASL still has a grammar right? Parts of speech (so to speak, er say, er... you know) and all that? Can you not explain in ASL what the letter a looks like? That it's vowel? that it's the first letter of the English alphabet? etc...?

                It's like music, you'd think explaining something like a trell [sp?] or grace notes would leave the audience mystified, but it can be done. Even though English has no concept of a grace note, or staccato, or accented (forte), or lagato, or etc... Most people learn music by a

                • Sigh... I'll continue to feed the trolls...

                  ASL still has a grammar right? Parts of speech (so to speak, er say, er... you know) and all that?

                  Of course ASL has grammar; do you understand the concept of language?

                  Can you not explain in ASL what the letter a looks like? That it's vowel? that it's the first letter of the English alphabet? etc...?
                  How do you think 5 yr olds are taught to write and read?

                  You learned English phonetically, as do most hearing children. What does a vowel signify to someone who neit

                  • I'm not trolling, I just refuse to accept that deaf kids can't learn to read.

                    There is more than one way to learn a language than just hearing it. I heard pictures and moving pictures work well too... I never said it was easy. Hell, it's not easy to learn to read/write even as a hearing person.

                    I guess cuz deaf kids are inferior they can't learn to read like us hearing enabled folk [that was sarcasm].

                    Tom

    • Re:TTY? (Score:5, Interesting)

      by KokorHekkus (986906) on Tuesday February 13 2007, @06:55AM (#17995554)
      The deaf texts a lot as well - one swedish article had a couple of deaf teenagers commenting on 3G phones and they used to send 500-1000 SMS per month. But a quick email exchange isn't the same thing as a actual live conversation (at least not for most of us)... just consider how the time gaps with texting/email makes it harder to judge the other persons mental state.

      Another interesting trial project going on now in Sweden is "Translator in a pocket". It allows a deaf person to call a sign language translator who translates using the phone. Very useful for anything where you need a direct conversation with a hearing person and you couldn't plan ahead to get a translator and don't want passing notes (or what they'd use). Btw, 3G phones are very popular here in Sweden with the deaf and especially with the teenagers. I've heard numbers that something like 80% of all deaf teenagers have videophones.
      • From the Phone Losers of America [phonelosers.org]:

        EPISODE #3 - Deaf Relay Operators

        This episode features my new co-host Mary, a relay operator. It also features songs, skits, messages, commentary, commercials and raps all performed by deaf relay operators. It's approximately 17 minutes long and the download is 15,597 kb. Click here to listen to it.

        Most of the relay voicemail messages played are more than 10 years old. The message involving the terrorist blowing up a commuter airline, done in 1994, was left on my own voicema

      • Re:TTY? (Score:5, Insightful)

        by KlaymenDK (713149) on Tuesday February 13 2007, @06:58AM (#17995572) Journal

        What is the benefit of this technology over TTY or using text messaging?
        I don't know what you or the GP means (in a mobile phone context) by TTY, but there is one definite benefit: immediateness. That is, lack of latency in message delivery.

        When you send text messages back and forth, there's a delay with every delivery. For the equivalent of speech, this would be like calling the moon. Plus, you have to go into the inbox and open new messages all the time -- not very conversation-like or, for that matter, IM-like.

        • Re: (Score:3, Interesting)

          by zakezuke (229119)
          When you send text messages back and forth, there's a delay with every delivery. For the equivalent of speech, this would be like calling the moon. Plus, you have to go into the inbox and open new messages all the time -- not very conversation-like or, for that matter, IM-like.

          I agree with the delay... but with T-mobile is it rather IM like on my nokia 6800 and 6010. In fact there is IM support. Again, speaking only for T-mobile... there seems to be two systems for IM... one is via a relay. It's rather t

          • Re:TTY? (Score:4, Insightful)

            by gEvil (beta) (945888) on Tuesday February 13 2007, @07:20AM (#17995682)
            Well it seems the same effect can be achieved through a video conference...

            And that's essentially what this article is about. Rather than using full-bandwidth video communication, they're trying to develop a compression algorithm that is better suited to signing (ie, capturing only the primary hand motions).

      • Re: (Score:3, Informative)

        by thevil (602459)
        The benifit is that sign language is the first language for a lot of people.
        English is their second language.
      • Last I checked BBC sells it's feed to other broadcasters. They'd be stupid to just give it away.

        That and most of the rest of the planet only gets BBC World [the news] not BBC 1-4. :-(

        Tom
      • As someone who speaks English and French, can express ideas in a half dozen programming languages, and can read music ... I say so what.

        What do you think a musical score is? Bunch of meaningless symbols, lines, dots and squiggles [to the untrained eye].

        I know I'm talking out my ass, but I really have a hard time believing that deaf children cannot be taught to read a written language. Kids are very versatile and also have all the time in the world to study.

        Sure, maybe an a deaf adult who never learned to
      • Not all deaf-from-birth people end up being raised on sign language.

        I'm born deaf. I was raised on written and spoken English, known as "oral education".

        I do know sign language, but not until I had entered high school. By then, my understanding of the English language was quite solid.

        -Cyc
    • Mod parent up, I was gonna write the same thing but searched to see if anyone wrote it first and well, yeah what he said They did it in babel.

    • Re:Sign language and speech faster than typing? (Score:4, Informative)

      by mr_matticus (928346) on Tuesday February 13 2007, @07:49AM (#17995850)
      Either you talk too slow or you've broken some land-speed records for typing on handheld devices. Typical English conversation is roughly 200 words per minute. Most of the population can't type faster than about 60 words per minute on a standard keyboard, let alone a cell phone-sized thumbpad.

      Even if you type at double that (120wpm), you're still typing slower than you speak. As for the input device, how would you go about making a pocket-sized keyboard as efficient as a desktop version (which you can put down and use all fingers to type--no such possibility with a cell phone)? Having to have the physical input device AT ALL *is* the problem to be fixed here.

    • Umm. I type as fast as I generally speak. I *can* speak faster, but then, I *can* type faster too, if I don't have to stop and think what I'm going to say. I imagine signing would be similar. So I would think text messaging would be just as fast. Unless the problem is that it's hard to type on the available input device. In which case, fix the input device. I don't guess there's anything _wrong_ with developing technology to allow sign language to be transmitted over the cell phone network, but it seems l

All trademarks and copyrights on this page are owned by their respective owners. Comments are owned by the Poster. The Rest © 1997-2009 Geeknet, Inc.