Forgot your password?
typodupeerror
Input Devices Cellphones GUI Software Technology

Developing "Eyes-Free" Gadgets and Applications 85

Posted by Soulskill
from the translating-web-sights dept.
The New York Times is running a story about Google engineer T. V. Raman, who lost his vision at age 14 but didn't let that stand in the way of his interest in technology. In addition to modifying a version of Google's search engine to give preference to pages that were more compliant with accessibility guidelines, Raman is now working on making cell phones easier to use without needing to look at them. "Since he cannot precisely hit a button on a touch screen, Mr. Raman created a dialer that works based on relative positions. It interprets any place where he first touches the screen as a 5, the center of a regular telephone dial pad. To dial any other number, he simply slides his finger in its direction — up and to the left for 1, down and to the right for 9, and so on. If he makes a mistake, he can erase a digit simply by shaking the phone, which can detect motion." Raman and a co-worker, Charles Chen, are also attempting to extend various phones' ability to read back scanned text to include signs that are anywhere in the phone's field of view.
This discussion has been archived. No new comments can be posted.

Developing "Eyes-Free" Gadgets and Applications

Comments Filter:
  • How... (Score:1, Insightful)

    by MrEricSir (398214)

    But how does he play Snake?

  • by Anonymous Coward

    See
    http://news.bbc.co.uk/1/hi/magazine/7807217.stm [bbc.co.uk] for an inspirational story by a blind man who got to one of the most powerful positions in the UK govt (the fact he was awful there had very little to do with his blindness).

    • by TheLink (130905)
      Many people in high government positions are disabled one way or another.

      For instance some have congenital absence of integrity or conscience.
  • You could probably translate different things about the text into different audio cues. Bigger text? Louder reading. Stereo headphones could also make the reading come from the direction of the sign. As image recognition gets faster and more accurate, I'm sure a number of different audio cues could tip a blind user off with clues about the environment. I am sure the usefulness of this will go well beyond blind users.

    • Re: (Score:3, Interesting)

      by hedwards (940851)

      Or we could admit that the current crop of iPhone inspired phones are a tremendous leap backwards. Seriously, even if you're not blind, the fact that you have to pull it out of your pocket to use is a pain in the ass. Makes me wonder what the point of wireless headsets really are if you ultimately still have to look at the phone to use it.

      I'm not sure any of that other stuff is going to be helpful if companies are exercising in a pathological hatred of necessary buttons.

      • Re:Reading Signs (Score:5, Interesting)

        by djupedal (584558) on Sunday January 04, 2009 @01:44PM (#26321595)
        >Seriously, even if you're not blind, the fact that you have to pull it out of your pocket to use is a pain in the ass.

        Yeah, because technology such as 'hands-free Bluetooth' hasn't been invented yet on your planet.

        When I'm using an earwig or in the car, the phone stays in the pocket...everything, from connecting to the car to taking calls happens automatically or via voice control - what a country.
      • by Fluffeh (1273756)

        Makes me wonder what the point of wireless headsets really are if you ultimately still have to look at the phone to use it.

        Well.... duuuuhhhh... it's there so that you can check your called ID and dodge the call from the GF while you are out drinking with the boys!

    • Then there's also the research going into piping images from cameras directly through the optic nerve. Wont work for everyone but it's something.

    • by MrMr (219533)
      Bigger text? Louder reading.
      ALL CAPS FOR SHOUTING!
      obviously.
    • The most interesting part of having the volume of the speech proportional to the size of the text would be in sales ads. Things like "on sale now" or "bad credit is not a problem" would be yelled out while all of the fine print would be whispered and almost impossible to hear.
      • by Meccanica (980734)

        The most interesting part of having the volume of the speech proportional to the size of the text would be in sales ads. Things like "on sale now" or "bad credit is not a problem" would be yelled out while all of the fine print would be whispered and almost impossible to hear.

        That's certainly how they do it on the radio.

    • When travelling around mainland europe, some friends and I bashed out an idea along these lines, except that it would translate signs into your language of choice. Ideally it would get its translations over the net, using Babelfish or whatever, and it could even use your current location to decide what default language it was translating from. Any app I wrote would probably be slow as buggery for doing all that OCR stuff on an N95, but the biggest flaw in the plan was ... roaming data charges/access. Not
  • by MrCrassic (994046) <deprecated&ema,il> on Sunday January 04, 2009 @01:18PM (#26321407) Journal
    I wish I could remember the name of this device, but it was essentially a MP3 player with no screen; just directional buttons and voice-based navigation. It was manufactured for blind users, and it worked wonderfully. It could even read text files and accept CF cards for expansion!

    I think that a cell phone with just buttons on it and braille lettering would suffice, provided that the voice navigation is really good. An added advantage is that having no screen can make for very thin and attractive devices, if aesthetics is something of a priority for them.
    • by rusty0101 (565565) on Sunday January 04, 2009 @02:07PM (#26321769) Homepage Journal

      One of the down sides to hardware that has no screen on it is that there is not a sufficient market for general use, and as a result the cost to the consumer is significantly higher. Braile readers for example end up being hand made because there isn't a sufficient demand to mass produce them, but the result is a display that's one or possibly 2 lines of text, that costs a couple hundred to a couple thousand dollars to produce.

      Part of the idea of what Raman is doing is taking existing consumer hardware and applying software solutions to make them more usable to the blind.

      A similar solution would be to use a cell phone to convert the words a person is speaking into text that a deaf person could read. You could build into it language recognition and translation and the deaf person on the bus next to you on your next tour of Europe, Japan, China, etc. may get more out of what is going on around you than you do.

      A screen free MP3 player may sound like something that a blind user would appreciate, especially if the control hardware, and the audio menus were well designed. The big problem becomes one of what all the device can do. Adding a camera to a device like this seems somewhat counter intuitive from a consumer goods perspective, even if the hardware cost were just additional pennies. How about gps location hardware. Again just pennies, but to build it into that MP3 player doesn't seem to make all that much sense, does it. Add in a compass and an accelerometer, and you have a navigation aid that a blind user could use to get almost anywhere with.

      The thing is that pretty much everything described is in a G1, so really adding usability for the blind is primarily a matter of plugging into the user interface with something that converts text to speech and where possible images to text. Both of those are partially solved problems. The part that's still difficult is getting it all tied together, and making available sufficient processing power to make it operate cleanly.

      • Re: (Score:2, Insightful)

        by mac.man25 (988406)
        While the iPod shuffle is not a particularly powerful device, it does allow the user to use a well supported device that most probably is not going anywhere soon. The iPod shuffle in my mind would be very useful if what you wanted was just music.

        However, If was blind, I might not want an MP3 player because I rely so heavily on my sense of hearing to get around. If I encased myself into my own musical world, I might miss something I need to hear.
        • My 82-year-old mother has lost all of her vision but a bit on the periphery (macular degeneration) and as her blindness set in she moved from large-print books to books on tape and on CD. But eventually those devices became impossible for her to use. Last year we got her a shuffle - spent about an hour with her teaching her how to use the no-screen device and she's been very, very happy with it ever since. We load it up, she can navigate within and between her books with relative ease. The darn thing just
      • by MrCrassic (994046)
        That kind of device won't be marketed towards that segment, nor should it be. Most devices that are designed for the blind are mostly utilitarian in that aesthetics take a step back but person-device interaction is highly emphasized.

        Take a look at some of the internet devices out there for the blind. They are computers in their own right, but the only interaction available is through the keyboard (and audio).
    • I wish I could remember the name of this device, but it was essentially a MP3 player with no screen

      I just want to point out that when a blind person walks down the street, the last thing they need is music coming out of their headphones, blocking noises from the environment.

      Now, if you manage to integrate effective guided GPS navigation for pedestrian walkways into such a device, then you're in business.

    • by beetle496 (677137)

      I wish I could remember the name of this device, but it was essentially a MP3 player with no screen; just directional buttons and voice-based navigation. It was manufactured for blind users, and it worked wonderfully. It could even read text files and accept CF cards for expansion!

      I have never heard of any MP3 player that works from voice commands, so I would like to hear more about that! Or do you mean synthesized speech based navigation? If so, you may have been thinking of RockBox [rockbox.org]. Which is FOSS tha

  • Brilliant (Score:5, Insightful)

    by Skiron (735617) on Sunday January 04, 2009 @01:20PM (#26321421) Homepage

    "Since he cannot precisely hit a button on a touch screen, Mr. Raman created a dialer that works based on relative positions. It interprets any place where he first touches the screen as a 5, the center of a regular telephone dial pad. To dial any other number, he simply slides his finger in its direction -- up and to the left for 1, down and to the right for 9..."

    So simple yet so brilliant. There is so much tripe published about 'innovation' (usually Microsoft), yet I think this is the first time _I_ can use this word properly.

    Well done Mr. Raman - truly brilliant.

    • Re: (Score:3, Insightful)

      by Linker3000 (626634)

      I was coming here to say pretty much the same thing - it's a great and simple idea that would even be of use to people with full sight. I often have problems 'dialling' a number on the touch screen of my HTC TYTN II - 'fat fingers' and all that.

      • Re: (Score:2, Informative)

        This isn't innovative - see mouse gestures, pie menus, even context menus that pop up where you first click, and not in a predefined place.

        Now if this guy Raman had instead invented Raman Noodles ...

        • Re:Brilliant (Score:4, Insightful)

          by Linker3000 (626634) on Sunday January 04, 2009 @01:37PM (#26321553) Journal

          Maybe true - but does any touchscreen phone currently on the marker have this damn useful feature?

          • All regular touchscreens (not phones) do. You can configure them so that a "double-tap" opens, for example, the context menu right under your finger, or launches a specific app with a specific interface at your fingers' location.

            Calling this innovative is like calling a blank white board innovative because "you can write anywhere - you're not constrained by the lines."

            In other words, this is not an "invention" - it fails the "obviousness" test.

        • Try doing that when you are blind with the mouse.

          What he has done is combined touch with an interface that configures itself to the touch, so he knows where his finger is, and at the same time knows where (whatever) the touchpad hotstops are.

          • This is no more different than popping up a window at an arbitrary place on a touch-screen. In other words, it's over a decade old.
        • Innovation
          1 : the introduction of something new
          2 : a new idea, method, or device

          The fact that you don't find it interesting, or you can think of vaguely similar concepts in unrelated mediums does not discount this idea and implementation from being innovative. Shaking the phone is relatively innovative and the whole package is somewhat interesting.

          That said, the device itself seems unlikely to really solve his problem of making cell phones easier to use without looking at them. I get the feeling
          • by Meccanica (980734)

            I get the feeling that the system he's got outlined will take a lot of time to get used to and still require the occasional glance at the phone.

            Blind. He's blind. He can't see. The man developing this is unable to look at the phone.

            • I get the feeling that the system he's got outlined will take a lot of time to get used to and still require the occasional glance at the phone.

              Blind. He's blind. He can't see. The man developing this is unable to look at the phone.

              That's okay. In Soviet Amerika, phone looks at YOU!

              Also, being legally blind doesn't mean being completely sightless.

              If he makes a mistake, he can erase a digit simply by shaking the phone, which can detect motion.

              Old technology. See Etch-a-Sketch.

              Look, this might

          • So Merriam Webster doesn't have the same definition as the patent office. Big deal.

            Also, this is not a "new idea", not a "new method", and by itself is not a "new device", so it fails.

            This is no more innovative than my grabbing a seat at random in a restaurant and having the staff serve my meal at that particular table instead of another one. They will then place everything (knife, fork, plates, meal) in the same relative starting position, just that the starting position is changed based on which ta

    • Re: (Score:3, Insightful)

      by mishehu (712452)
      Is the whole thing really brilliant, or is what Mr. Raman doing just an elaborate work-around for what was essentially a usability step backwards? One thing that I like about my Treo phone is that it has a combination of a touch screen AND has real buttons that I can feel with my fingers and not have to look at the screen if I don't want to. Hell, I can call into my office phone system, put in my voicemail authentication codes, and listen to my voicemails without ever having to look at the phone...
    • Re: (Score:1, Funny)

      by Anonymous Coward

      It's brilliant, however I doubt Mr. Raman will ever surpass his soup patent.

    • by Nuitari The Wiz (1123889) on Sunday January 04, 2009 @02:48PM (#26322069)

      How a bout having a set of predefined buttons with a small bump so that the number 5 can easily be identified. The buttons could be placed in some sort of logical order, let's say in ascending numbers, 3 each row. We could also add * # Send and End as buttons.

      It might look something like that:
      S E
      1 2 3
      4 5 6
      7 8 9
      * 0 #

      What would really be cool and useful too is to make sure that when someone presses a button it also feels like its been pressed.

      As an aside, a cool feature would be the cellphone speaker phone saying the number that was pressed as someone dialed. _That_ would be useful.

      • As an aside, a cool feature would be the cellphone speaker phone saying the number that was pressed as someone dialed. _That_ would be useful.

        But phones already do that. They play the tone pair [wikipedia.org] for each digit that the user dials.

      • by Rah'Dick (976472)

        You mean with real keys, like any old non-touchscreen phone? Next thing you say that office applications should be client binaries, not run off the web! ;-)
        The only new thing would be the phone saying the number you just pressed, but you kinda already have that, too - DTMF tones.

        • Re: (Score:2, Informative)

          Yeah exactly ;-)
          I was actually being very tongue in cheek.

          As for the dtmf, I meant something a bit more user friendly, like having a digitized voice that would say it blind people.

          It's not like cellphones require dtmf to dial out...

      • by HTH NE1 (675604)

        As an aside, a cool feature would be the cellphone speaker phone saying the number that was pressed as someone dialed. _That_ would be useful.

        Yeah, it may get cellphone users to step into special closets to make their calls to prevent others from knowing what numbers they're dialing. And then maybe they'll just stay there until the call is finished.

        In fact, they could just put a wired phone in there so people wouldn't have to carry a cell phone and just charge a modest fee for its use.

    • by abigsmurf (919188)
      "Usually Microsoft"? Because Apple are known for being incredibly modest and underplaying any features in their products right?
    • Simple yet brilliant huh?

      So, would you say it is patent worthy? Or was it obvious to somebody 'skilled in the art'?

      • by HTH NE1 (675604)

        Necessity is the mother of invention. Patents are the motherfuckers of necessity.

    • Re: (Score:2, Insightful)

      by tdi65 (1444423)
      I would be interested in this technology even as a person with eye sight. I can't stand blue tooth technology, no matter how much I pay for an earwig I get the "you sound like you are in a tunnel, is everything ok?" and conditions have to be stellar for voice activation to work without me finally just screaming expletives at high volume into my cell phone while driving. And forget about using touch screen phones in direct sunlight outside. As an outside sales person with no choice but to use my company c
    • by hobbit (5915)

      You've really never come across something more innovative than this?

  • afaik there are plenty of phones on the market that have speech recognition such as speaking 'mum' and it will auto dial, ive never used them so i dont know what sort of quality they are at atm but that seems to me like it would be a better way of doing things, since it also means you wouldnt need to memorise every number you might need

    are there any blind /.-ers whom might be more qualified to give an opinion?
    • Re: (Score:1, Interesting)

      by Anonymous Coward

      Voice recognition is annoying enough for sighted people. For the blind who place greater reliance on audio feedback from the device it's a pain in the ass. Every time you say something to the phone you'd then have to stop speaking in order to listen to it respond. Or keep talking until the end and then try to fix all the errors through voice control... fun, fun, fun.

    • by LinkX39 (1100879)
      Unfortunately voice recognition, at least in my experience, is very hit or miss. First of all, you still have to hit a button to activate voice recognition, then you have to say the name you wish to dial, then you have to keep saying no to all the names the phone thought you said.... And that's only if the number is programmed into the phone already. My experience with voice-to-number dialing has been even worse than voice-to-contact. You'd be amazed at what numbers the phone can confuse with one another.
  • I suspect that this technology will be used more by drivers and teens than the handicapped.
    • by nedlohs (1335013)

      Of course it would. A phone with keys with braille (or raised numbers/letters or whatever) on them would be better for a blind person - why would they want ab LCD screen covering the whole device even if it is a touch screen?

      The article does mention that, something along the lines of the guy likes to think about things in terms or what if the user can't look at the screen right now, rather than what if the user was blind.

    • by Thanshin (1188877)

      I suspect that this technology will be used more by drivers and teens than the handicapped.

      Which would be a good thing. Just as good as controlling by speech: lifts, lights, air conditioning, vending machines, etc.

      It's much safer to rely on the general public's lazyness than to rely on solidarity.

  • Why not.. (Score:2, Informative)

    by barcrawler (913684)

    If we're working on making a touch screen phone more accessible to the sightless, why not ditch the screen entirely and replace it with a tactile display capable of adapting to the needs of the user? This would make it possible to still have your email or even text messages right at your fingertips - literally!

    Here's a prototype that I'm sure could be improved upon and made portable given the right amount of funding.

    http://www.nist.gov/public_affairs/factsheet/visualdisplay.htm [nist.gov]

    • Why don't we have more purpose-built communication solutions for the handicapped? It's simple, really - economies of scale. Take a look at the god-awful, drm-wrapped, clunky, super expensive CD players for the blind and you'll understand. Those $220 devices do a much crappier job than the $49 Shuffle. The (theoretical) brilliance of Raman's design is that it makes sense for MORE than just the blind, which means that it has the potential to a much better device in ways that go beyond the interface.
  • There are some problems with the method he's chosen. Most notably 7,9,1,3. It's pretty hard to configure the bias for diagonals so that people don't accidentally do a diagonal instead of a direction or vice versa. It may seem easy to do but when you're dealing with an inch of travel, a slight amount to the left or right can make a huge sifference to the angle. There are also issues with phone alignment in the hand.

    I think Ultimately the best solution is embossed numbers on the phone or other tactile metho

  • Where will this end? First they'll make a cellphone that has no eyes and the next thing you know they'll make one with no spleen.

    I better hold onto my Nokia SpleenMaster 5000, it might be the last of its kind.

  • That's pretty cool (Score:3, Interesting)

    by geek (5680) on Sunday January 04, 2009 @01:56PM (#26321679) Homepage

    I went to a college that catered heavily to people with disabilities, primarily the deaf but also a large number of the blind. It occurred to me early on that a great deal of the tech they use is developed by people without disabilities and then tested on people with them.

    I think Mr. Raman is pretty unique in the sense he's able to develop like this as a blind man. That said, I'm curious why voice recognition wasn't considered the better option?

    • Re: (Score:1, Informative)

      by Anonymous Coward

      In my opinion (take it for what it is), this is useful for different situations. Ideally, you could have a device capable of this type of dialing and voice recognition. There are places where voice recognition is not the best choice. Say you're in a public place & there's a lot of background noise. The device isn't necessarily able to pick out the commands from the other stuff. On the other hand, it may be much easier to utilize voice recognition then to fumble around with your phone trying to dial wit

    • If you don't mind, what college did you go to? My younger sister is a senior in high school and is blind, she might appreciate the info.
  • What's the difference between this and swype [slashdot.org]?
    • Re: (Score:3, Interesting)

      by rusty0101 (565565)

      swype is intended as a replacement for an entire keyboard, and presumes that you are reasonably close to the first letter in the word.

      One way to make swype work like this would be to start with a 'common' character on the keyboard and draw from there. Preferably central to the keyboard. Another alternative would be to build a 5x5 grid of letters, doubling the c and k characters into one cell of the grid, and using that as a swype board. (It would fit on a screen better as well) That layout is what the tap c

  • ... why would a blind person even get a touch-display phone? Probably not for the sleek UI, or?

  • ... as long as it begins with 5?

    • A '5' is the "orientation tap" and any numbers following are the numbers to dial. The phone could say each number as it's tapped and a "swipe" across the screen could delete the most recently tapped digit.

      A "fist pounding" on the screen could indicate high user-frustration. The machine could then re-boot to force a "time out" on the user.

      Users can be trained.

    • No, every second digit has to be a five, eg 595151. Also, the number may not contain any zeros, because you can't dial them...
    • by HTH NE1 (675604)

      The parent should get one Funny mod.

      But no, a tap would get a five anytime. Strokes in 8 directions get the other digits other than zero. A circular stroke gets zero. How to get # and * though, I don't know.

  • Even with a high-end noise-canceling Bluetooth headset and a phone with voice recognition, the interface sucks.

    First, the Bluetooth headset has one button, which both originates and terminates calls. Pushing it generates a tone, but almost a full second after pushing the button. The same tone is used for connect and disconnect. Dumb.

    Starting from the idle but synchronized state, pressing the button yields, after a few seconds, a tone, and then, after another five seconds, the message "Say a command"

    • I generally agree with you. That said...

      Considering that hands-free operation is mandatory in California for phoning while driving, one would think this would be done better.

      I was steering my vehicle with my knees long before "hands free operation" was made mandatory in California.

      Any movement to legislate "eyes-free operation" is going to make driving even more challenging, as the braille lane markers are missing from a lot of California's streets. What's a person to do? Get one of those talking GPS n

      • by drosboro (1046516)

        Oh, but make sure you don't stick your new talking GPS to your windshield. That's illegal in California too... :)

  • The spanish organization ONCE (Organización Nacional de Ciegos Españoles - Blind Spanish Men Organization) has a group that also develops applications for blind people.

    Website (in spanish) http://cidat.once.es/ [cidat.once.es]

    Some applications (which you can download trial versions):
    MOBILE SPEAK POCKET
    http://cidat.once.es/home.cfm?id=184&nivel=2 [cidat.once.es]
    You can hear the text of the text in a windows mobile pda

  • I wonder why they don't make any mobile phones with chorded keyboards. These would also be easy to use without looking at them, and it would also be easier to type using just the hand you're holding the phone in.
  • This is an awful lot of technology just to avoid the fact that all one needs is an old fashioned tactile keypad with real buttons in order to do the same thing.
  • Louis Braille [wikipedia.org], inventor of the alphabet glyphs readable by blind people, was born exactly 200 years ago today.

  • What all this really means is those without a visual handicap will probably take advantage of such advances in technology to do more computing while driving. I'm certainly guilty of being able to create, write and send text without looking. The advent of the cell phone internet browser only promoted the ability to browse porn while driving, imho. Now if only they could translate THAT technological advance adequately for our visually impaired friends. Talk about a real contribution to society.

"Well, social relevance is a schtick, like mysteries, social relevance, science fiction..." -- Art Spiegelman

Working...