Please create an account to participate in the Slashdot moderation system

 


Forgot your password?
Close
typodupeerror
×
Communications Handhelds Hardware Science

Sign Language Via Cell Phone 151

Posted by kdawson from the can-you-see-me-now? dept.
QuatumCrypto writes "A project is underway at the University of Washington to enable real-time sign language communication via cell phone. Because of the low-bandwidth wireless cell phone network, a new compression scheme is necessary to capture only the bare essential components of signing to minimize data transfer. Although text messaging is a viable alternative for everyone, signing — like speech — is a much faster and more convenient form of communication."
This discussion has been archived. No new comments can be posted.

Sign Language Via Cell Phone More

Sign Language Via Cell Phone

Comments Filter:

  • Video calls (Score:3, Interesting)

    by dotancohen ( 1015143 ) on Tuesday February 13, 2007 @06:14AM (#17995356) Homepage
    I've already seen sign language being used over video calls. Then again, as one who volunteers with autistic children, I've seen a lot of super-use of technology and hands...
    • Just out of curiosity, how does this work? Did they get someone else to hold the phone for them, or can ASL work with only one hand?

      • Re: (Score:2, Informative)

        by dotancohen ( 1015143 )
        Yes, we hold the phone for him. I've also got a Nokia 6280 with video calls. The video calls at 0.46 NIS/m are cheaper than regular phone calls at 0.67 NIS per minute. The boy is question is not only autistic but also mostly deaf. What's interesting is watching young kids talk on the phone. Even on a non-video call they nod yes and no.

    • I've seen a lot of super-use of technology and hands...
      ... and we're all very familar with that around here.

  • They're focusing on video... (Score:5, Insightful)

    by Mr2001 ( 90979 ) on Tuesday February 13, 2007 @06:16AM (#17995368) Homepage Journal
    But there's another problem with using sign language via cell phone. Look at the screen mock-up on that page - it shows the signers from the waist up. If your phone is far enough away that it can capture your whole body, how are you going to see the screen?

    Also, they claim "The current wireless telephone network has inadvertently excluded over one million deaf or hard of hearing Americans", but it's easy to get a cell phone that supports TDD [phonescoop.com], just like a wired phone.

    • Re: (Score:1)

      by jonadab ( 583620 )
      > If your phone is far enough away that it can capture your whole body,
      > how are you going to see the screen?

      It doesn't need to capture the whole body. Waist-up is adequate for ASL and probably most other sign languages. Deaf people want to communicate while seated, as well as while standing, so gestures involving the lower body are not used much.

      Of course, people with impaired vision might have trouble seeing it even at a distance that captures from the waist up only, but the goal here isn't to sol

      • Re: (Score:2)

        by Mr2001 ( 90979 )

        It doesn't need to capture the whole body. Waist-up is adequate for ASL and probably most other sign languages.

        That's what I meant. If the camera is far enough away that it captures you from the waist up, that's pretty far. Holding my cell phone at arm's length, it only sees me from the top of my head to just above the bottom of my ribcage.

        I suppose you could fit the camera with a special lens to give it more vertical range... but that leads to another issue: if you're holding the cell phone at arm's length, how are you going to sign with two hands? If you need to set it down on a tripod or something, then you can'

    • You don't understand. TDD (also called TTY among Deaf) is not a primary mode of communication. Just as Hearing people's primary mode of communication is talking, Deaf people's primary mode is sign language. In this regard, the Deaf population has been left behind.

      What's happening now is that the existing network is being retrofitted for something that it was not originally designed for. In the U.S., it's fairly impossible to achieve a truly useful ASL communications device with the lack of high speed cellul

    • Re: (Score:2, Interesting)

      by accessbob ( 962147 )
      I was at a presentation of their paper on this in Portland last year: http://portal.acm.org/citation.cfm?id=1169001&jmp= cit&coll=ACM&dl=ACM&CFID=14265233&CFTOKEN=82641255 #CIT [acm.org] From the abstract: "...techniques that exploit the visual nature of sign language. Inspired by eyetracking results that show high resolution foveal vision is maintained around the face, we studied region-of-interest encodings (where the face is encoded at higher quality) as well as reduced frame rates (where few

    • Re: (Score:2)

      by necro81 ( 917438 )

      it's easy to get a cell phone that supports TDD

      That's all well and good, but that requires carrying around a TDD keyboard in addition to the cellphone. Those things aren't small. It also requires that the receiving party also have a TDD, unless the cellphones know to display the TDD text on their tiny screens.
      • Well, yes, it would require carrying a keyboard. But they make some pretty small and light keyboards these days, so it's not going to be that big of a deal.

        And the receiving party doesn't need a TDD, they can use their voice - the TDD relay service is free.

      • Re: (Score:2)

        by ncc74656 ( 45571 ) *

        it's easy to get a cell phone that supports TDD

        That's all well and good, but that requires carrying around a TDD keyboard in addition to the cellphone. Those things aren't small. It also requires that the receiving party also have a TDD, unless the cellphones know to display the TDD text on their tiny screens.

        With something like a Treo, it should be possible to emulate a TDD within the device. It has a suitable display and keyboard, and it's small enough to carry around. All that's needed is an

    • I learned Sign Language (ASL) and use it with my Deaf[1] patients. Parent is right that you need to include the body from the waist up. Many people don't realize that, besides the handshapes, there are three other components to Sign Language, which lead to certain requirements in video streaming for Sign Language:
      1. motion; for example, for "insurance" the hand shakes rapidly side to side, whereas for "infection", it makes a small circle. Otherwise the handshape is the same. So Sign Language video needs t

  • How do you hold the cell phone? (Score:4, Insightful)

    by niconorsk ( 787297 ) on Tuesday February 13, 2007 @06:17AM (#17995372)
    The technology for this is very cool and all, but I don't see it as very applicable to use with cell-phones. As far as I know requires the use of both hands, so you would have to put down your phone in a way that you can be seen and you can see the screen and lastly without holding it. This seems like an impossible proposition. But the technology in its own right could be very interesting, at least for desktop video-conferencing units.
    • I know requires the use of both hands, so you would have to put down your phone in a way that you can be seen and you can see the screen and lastly without holding it

      That was my first thought as well, but it will be nice to have the software ready for when little wireless spec displays & cameras are available (if they haven't thought of a solution already).
    • You just set the phone flat on the table in front of you and angle up the top part, which contains the camera and the display. There is nothing magic about this - deaf users in Sweden do this all the time with their 3G phones.

      Some people also hold it with their left hand in front of them and sign with their right. Although signed languages use both hands, deaf people can hold a perfectly intelligible conversation signing with just one hand.

  • Language-agnostic? (Score:3, Insightful)

    by Max Romantschuk ( 132276 ) <max@romantschuk.fi> on Tuesday February 13, 2007 @06:17AM (#17995374) Homepage
    I hope this compression scheme won't be tied to the semantics of a single sign language like ASL. There are plenty of other sign languages in the world, so hopefully this tech will be "language-agnostic", so to speak.
    • Well, the main page of the site have two pictures showing examples of skin detection algorithms and motion vectors as the main tools, so that should be pretty language independent. Personally, skin detection algorithms bug me. They're fairly simple solutions for face and hand tracking and work well under the assumption that everybody has the same color of skin, but we all know this isn't really the case.
      • They're fairly simple solutions for face and hand tracking and work well under the assumption that everybody has the same color of skin, but we all know this isn't really the case.

        Well, there's a pretty simple solution to this problem. We just need to pass some legislation stating that you can't be deaf unless you have a certain skin tone.

    • Re: (Score:1, Funny)

      by Anonymous Coward

      ASL
      14/f/cali

      ...sorry, that was a reflex.

  • Makes sense (Score:4, Funny)

    by $RANDOMLUSER ( 804576 ) on Tuesday February 13, 2007 @06:23AM (#17995408)
    I often use sign language to people using cell phones while they're driving.

  • Hmmm (Score:2, Funny)

    by kitsunewarlock ( 971818 )
    I still think deaf people should communicate by getting to kick non-deaf people in the crouch. It works similiar to morse code, but with "crunches" and "squishes" instead of "dots" and "lines".

    But I'm one for giving handicapped people excuses to hurt the rest of us. It just seems fair. And I wear a cup.

    • Re: (Score:1)

      by jonadab ( 583620 )
      > I still think deaf people should communicate by getting to kick non-deaf people

      Somehow, I don't think that would improve their relations with the rest of society.

  • no subject (Score:5, Informative)

    by UnixSphere ( 820423 ) on Tuesday February 13, 2007 @06:53AM (#17995546)
    "What is the benefit over txt messaging?"

    Sign language is much faster obviously, and sign language is based alot on the user's emotions and how they use a certain sign or signs.

    But to answer the parent's question, none of the cell phone carriers offer a price break for deaf/hard of hearing users.

    BUT the deaf community is fond of using the t-mobile sidekick, all versions, because of the relatively cheap unlimited txt/data plan that comes with it. Sidekicks are almost dominant among deaf people. Some deaf tech sites and companies offer the sidekicks significantly cheaper to deaf users since it is so popular among them.

    • Re: (Score:2)

      by Aladrin ( 926209 )

      A while back, I had a neighbor who was deaf. I helped him and his hearing wife with their computer a few times for free. (And not out of pity, because I didn't know he was deaf when his wife asked for the help.)

      Anyhow, he also used a sidekick. Unfortunately, I know this because I found it in the parking lot, run over.

      He was a nice guy, but a little too eager to communicate with other people. He came across as simple because of it, but I don't think he really was.

    • T-mobile offers unlimited texting on any of their phones for 10 bucks, or 20 bucks for upto 5 phones on a family plan. (Or if you get lucky like me, you get offered unlimited texting on upto 5 phones for 10 bucks :D )

      T-mobile rocks the world in terms of alot of issues vs. other carriers.
  • No, what you need is a pair of wiimote-like gloves that you wear which are connected to a tiny robot monkey on the recipients cellphone that mimics your movements. The recipient, in turn, wears another pair of gloves which are connected to the robot monkey on your phone.

    OK, so instead of a robot monkey you could have a little animated monkey on your display, but a robot monkey would be better. Tiny robot monkeys is how Apple will implement it on the iPhone while the rest of the industry just has animated
    • The video call at a distance is just awkward. I suppose in certain situations even that can be valuable, but as an everyday thing I don't see it happening.
      I agree, wiigloves would be the way to go. The way that they show the video analysis is similar to what artificial intelligence geeks have been trying for years at to give robots sight - with little success. It's not what the sign looks like that's important, it's the motion, so video capture is totally unnecessary. Besides, motion capture data is far s
  • Although text messaging is a viable alternative for everyone, signing -- like speech -- is a much faster and more convenient form of communication

    Speech is flavored in languages, like text. So speech is not convenient at all if this is what they are saying. Otherwise, signing is not more convenient because only a small fraction of people already know it. I'm confused. Someone explain it to me.

    • In this context signing should be treated as a dialect. Because just like speech, the way the hands move can be used to indicate inflection (flavored) more easily than texting.

      As for not many people knowing sign. I would say that since cell phones are a more personal means of communication, most calls would be with friends and family who do sign.

  • They are flocking to the Sorenson VP-100 system.

    I cannot, for the life of me understand this, when there
    are so many video based chat sites on the net.

    All the deaf people I know have PC's. I met my first
    deaf friend on the old BBS's. In the text messages on
    FIDOnet.

    I would not want a deaf user signing while driving :P
    • Because the Sorenson VP allows deaf people to assign a phone number to it. So users can just dial the number directly to whoever they want to talk to that also has the same unit.

      The quality on web based video chatrooms/clients is also not very good, not sufficient for real time sign language. The Sorenson VP is free as well, so some deaf users might simply not have a webcam but they can get this unit for free.

    • I would not want a deaf user signing while driving :P

      It's like I learnt in Italy - you NEVER speak to an Italian when he/she is driving, because they are forced to take both hands off the wheel to reply to you!

    • Re: (Score:2)

      by woolio ( 927141 )
      I would not want a deaf user signing while driving :P

      That might not be so bad... The ability to hear and use one's hands for driving don't seem to do much good for the vast majority of the public. At least deaf people would be are used to it.

    • There is a simple reason why deaf people prefer the VP-100 and VP-200 over video chats: Ease of use. The Sorenson phone instantly boots up if it is turned on, allows you to plug in a flasher, so that you can see when it rings, there are no complicated menus and setups to navigate through, and making a relay call is just a button away. All these things may seem like little nits individually, but add them up, and there is just no contest between a computer and a dedicated device like the VP-100.

      Plus, being ab
  • > Although text messaging is a viable alternative for everyone, signing --
    > like speech -- is a much faster and more convenient form of communication."

    Umm. I type as fast as I generally speak. I *can* speak faster, but then, I *can* type faster too, if I don't have to stop and think what I'm going to say. I imagine signing would be similar. So I would think text messaging would be just as fast.

    Unless the problem is that it's hard to type on the available input device. In which case, fix the input

    • Re:Sign language and speech faster than typing? (Score:4, Informative)

      by mr_matticus ( 928346 ) on Tuesday February 13, 2007 @07:49AM (#17995850)
      Either you talk too slow or you've broken some land-speed records for typing on handheld devices. Typical English conversation is roughly 200 words per minute. Most of the population can't type faster than about 60 words per minute on a standard keyboard, let alone a cell phone-sized thumbpad.

      Even if you type at double that (120wpm), you're still typing slower than you speak. As for the input device, how would you go about making a pocket-sized keyboard as efficient as a desktop version (which you can put down and use all fingers to type--no such possibility with a cell phone)? Having to have the physical input device AT ALL *is* the problem to be fixed here.

    • Umm. I type as fast as I generally speak. I *can* speak faster, but then, I *can* type faster too, if I don't have to stop and think what I'm going to say. I imagine signing would be similar. So I would think text messaging would be just as fast. Unless the problem is that it's hard to type on the available input device. In which case, fix the input device. I don't guess there's anything _wrong_ with developing technology to allow sign language to be transmitted over the cell phone network, but it seems l

  • ...thump, whack, slap. Sorry!
  • I'm having trouble seeing any way of this working at all, for several reasons.

    First, cell phones are not known for capturing detail so well on their small screens. Is a phone, even a camera phone, capable of taking in a person's signing that well?

    Second, there's already a phone that's found common usage with the Deaf: The Sidekick [wikipedia.org]. Several members of my university's ASL club use it as a regular communications device. That and I think you can also implement TTY on it. It's nearly the most useful method

    • Re: (Score:1)

      by sim000 ( 721371 )
      Yes, camera phones with standard, already existing 3G video calls have sufficient quality that they can be used for sign language as it is. I understand this is quite popular in Sweden.

      What I have a problem with is why should we develop a single-use solution requiring new phones and what-not when we have an underused existing technology (video calls) that already work well for this purpose? 3G networks are becoming so common that it can't be a bandwidth issue either.

      • why should we develop a single-use solution requiring new phones and what-not when we have an underused existing technology (video calls) that already work well for this purpose?

        So that we can tune the codec for use with sign languages. I seem to remember that the voice codecs used on early mobile phones had to be retuned for use with Mandarin and other Chinese languages, which use a larger set of distinct phonemic tones than the Germanic and Romance languages against which the early systems were developed.

    • Sign language using deaf people overwhelmingly prefer to use video if it is available, for the same reasons that hearing people generally prefer a voice talk over having a text conversation. The only reason that the Sidekick and Blackberry are so popular among the deaf in the USA right now is that the US wireless networks are too slow to support signing.

      In Sweden and Denmark, where 3G is widely available, it is a whole different world. About *everyone* in the deaf community there uses wireless video on a ce
  • Seems like there are so many issues to overcome before this mode of communication would be realistic. Like many other posters here, I feel that texting would be more efficient and reliable. ... However, I'm not hearing-impaired, and my thoughts are based on that ...

    The important thing is that if hearing-impaired folks find a new (and maybe better for them) mode of communication, then that's a good thing, and more power to them!

  • Just hack Wiimote! (Score:3, Insightful)

    by 140Mandak262Jamuna ( 970587 ) on Tuesday February 13, 2007 @07:30AM (#17995742) Journal
    Another big chance for Nintindo. Can they hack the Wiimote to translate sign language to text?
    • Sign language is more in the fingers, the wii-mote would be useless here.

      Stop modding him insightful.
      • Well, the Wii can handle four wiimotes simultaneously. You need something like ten motion sensors to do sign language. But sign language to text wiimote will be covered by medical insurance as a "medically neccessary prosthetic device". True, gamers are willing to pay insane price for the "in" thing. But the profit margins in medical devices is an order of magniture higher than game consoles. For example the bluetooth ear pieces are being thrown in as freebies or being sold at scrap yard prices. But the hea
        • Now that's an insightful post. I'm sorry I doubted you, but you really spoke like you didn't know what you were on about before. I still don't think that the wiimote is a good basis for a sign-to-texting device (making one at all is still a great idea), but using a wii for physio seems like a good idea.

          There have been therapy-based consoles before, I seem to remember one for treating something like ADD that paused the game whenever the player stopped paying proper attention. If they improved the wiimote's s
      • ASL is more in the face/torso & body shifting. Fingers help, but they are not all of it.

    • Re: (Score:2)

      by pbhj ( 607776 )
      I think you're joking. But just in case, in BSL at least, sign language relies on nuance to form different words - like lip shapes, facial expressions, etc.. For example you can do a sign for "lemonade" and for "to f***" (in the sexual sense, not in the fsck sense) that differ only in the facial expression.

      Deaf kid (signing): Mom, don't forget to buy lemonade for Dad

      Mom: Wait till I get home you dirty little brat!

  • They've obviously not considered the impact this could have on driving while using your cellphone.

    Take a trip with me into the future & meet Marcie, Marcie is deaf & uses a sign language enabled cellphone.

    While driving on the highway (yeah, still no flying cars), Marcies husband signs to her over the phone that he wants a divorce so he can marry his secretary.
    She swerves slightly due to her shock.
    I, being the cautious driver I am ahead of her, look in my rearview mirror at the moment she hap

  • Videophones (Score:4, Informative)

    by Zarhan ( 415465 ) on Tuesday February 13, 2007 @07:44AM (#17995822)
    At least in Finland, in cooperation with a Finnish hearing-impaired association, there's been some projects with 3G video-phones. Yes - selling a phone to deaf people opens up a nice new market :). Anyway, as far as I know the experiences have been overall positive - and no fancy sign-language-specific codecs or anything, just a normal 64kbps video phone call and a camera phone.

  • Sign language over mobile works on 3G already (Score:4, Informative)

    by threepoyke ( 1063604 ) on Tuesday February 13, 2007 @08:08AM (#17995944)
    There is nothing new about this story. Sign language over mobile (cell) networks already works with regular 3G (UMTS) phones in Europe. Take a trip to Örebro in Sweden, which has a high concentration of hearing impaired due to a specialist education cent(e)r(e), and you'll see loads of teenagers using their 3G phones to talk using sign language. In the streets, on the bus, in cafes, everywhere. This article http://svt.se/svt/jsp/Crosslink.jsp?d=37482&a=5369 32 [svt.se] (in Swedish) from February 2006 even talks of the local social security services offering customer service to hearing impaired using 3G phones and sign language.
    • When I tested a 3G phone in Sweden, I was able to hold a conversation, but the frame rates were barely at the low limit of what is intelligible. While this is better than what we have in the USA right now, higher framerates or level of detail in the face would make comprehension much easier, and also cause less strain, as you need to concentrate less to understand the message.

      There is no reason that this technology can't be applied to 3G phones, as well, to make sign language conversations even better and m
  • A better idea might be something for the blind that vibrates out text messages in a morse code fashion. I expect this has already been done somewhere though. Gareth.
  • Hey, they are trying to leave us out again! First we can't listen in on them as they talk between cars, busses and trains. Now they want to use sign over the phone too? If we can't talk they shouldn't be able too either. It's only fair. OTOH, it would be nice to have a "finger" text key... to give someone else the finger.... maybe a talking milkshake giving the finger as recently seen in Boston?

    I'm being funny.... mod +1, funny.

  • As the father of a now adult deaf daughter I have seen the quantum leap of communication over the past 30 years. When my daughter was growing up the TTY was the only tool available to the deaf for communication. Each person needed to have a TTY and the speed was a blazing 48 baud using the old Western Union standard. Text pagers came along and relay services allowed the deaf to reach outside of the limits that existed. However, would you want to have an interpreter in the middle of every conversation yo
    • +5 insightful. I have a few good friends that work at Sorenson's(sp) Video Relay Service. Even the extremely high level interpreters have trouble with relay calls from out of our region as ASL as well as English change in different areas. Imagen a relay call between a hearing person from the south and a deaf person that grew up in england and knows BSL better than ASL and the interpreter is born and bread southern Californian. No mater how good you are, it won't be easy.

      On a side note, a deaf friend of

  • There's somthing very stupid with the concept of a sign language chat via cell phone. People are limited by the capabilities of a phone, and at the same time, given the capability, people will use it as they please. That means that no such 'sign language phone' exists (or will ever exist).

    Either a phone has video transmitting capabilities, or it doesn't. If it doesn't, then there's no hope of having a sign language chat (unless we use CGI to simulate it, which would be fancy texting). If it does, then optim
  • Siging is good for the deaf, but the blind have trouble using cell phone because they all react to input graphically instead of audibly.
  • I wonder why they wouldn't use Bluetooth glove controllers instead (like that Ultra Power Glove concept, except there's no need for feedback). That way you don't need to hold your cellphone away to be able to sign, you can pretty much just put it on the desk and sign and see the response at the same time. Small app would stream the accelerometers data to make wire model of other party's hands signing, and even regular EDGE speeds should probably be fine (ok, throw in compression if you want to). Most normal

  • The idea that text messaging is always an easy alternative for deaf people is actually not the case. It is easy in the sense that typing and reading are unaffected by the inability to hear, but text messages are in the written form of some oral language, such as English. For people whose hearing is profoundly impaired from birth, English or whatever the local oral language may be, is a foreign language. In the US, many congenitally deaf people have ASL as their native language. They learn English to varyin

    • Re: (Score:2)

      by geekoid ( 135745 )
      Then how do deaf people function in a work capacity if they can not read?

      I would also feel more comfortsble if everyone spoke english, because it is easier for me to communicate.

      Yes, it is entirely ASL and speaking are entirely different. To think the deaf people can't learn both is insulting.

      Not that I have anything against sending sign via phone, just the your argument flies inthe face of my experience with the deaf.

      • To think the deaf people can't learn both is insulting.

        No, its a recognition of a fact. Except when immersed in a language at an early age, people are not that great at learning languages. When you add to that the fact that those who are profoundly congenitally deaf can't hear English, it isn't at all surprising that it should be difficult to learn. How well deaf people know English varies enormously. Some are quite comfortable, others are not.

        Then how do deaf people function in a work capacity if they

    • Many deaf people will feel much more comfortable if they can sign than if they have to use text messaging.

      Well... TDD/TTY terminals are used by deaf users. They permit deaf people to use the phone. IM services have been accepted by hearing people. This is a case where both hearing and deaf can use the same thing. I can't speak for the deaf, but but any way you look at it, it's a practical solution.

      The real question would have to be put to the deaf community... whether they prefer typing or simplified vi

  • This is just stupid. I have already seen examples of deaf people signing to each other via cell phone. (See the movie, "Babel".) It's choppy, sure, but a 30fps video mobile was just announced which should do the trick.

    My point is, the technology for simply transporting full video is already becoming available for purchase. I don't see any need for a specialized codec.

  • A deaf friend calls me from time to time. She either txt msgs or uses her computer to IM a translator service here in Dallas. She gives them my name and number, and what she wants to say. They call me, tell me they are the service, who is calling, and what the msg is. They then text or IM her (depending on how she contacted them). We go back and forth that way. Slow, but functional. So far, we haven't tried talking dirty to each other via the service, but we've thought about it. :^)

Slashdot Top Deals

No spitting on the Bus! Thank you, The Mgt.

Close