Developing "Eyes-Free" Gadgets and Applications

The New York Times is running a story about Google engineer T. V. Raman, who lost his vision at age 14 but didn't let that stand in the way of his interest in technology. In addition to modifying a version of Google's search engine to give preference to pages that were more compliant with accessibility guidelines, Raman is now working on making cell phones easier to use without needing to look at them. "Since he cannot precisely hit a button on a touch screen, Mr. Raman created a dialer that works based on relative positions. It interprets any place where he first touches the screen as a 5, the center of a regular telephone dial pad. To dial any other number, he simply slides his finger in its direction — up and to the left for 1, down and to the right for 9, and so on. If he makes a mistake, he can erase a digit simply by shaking the phone, which can detect motion." Raman and a co-worker, Charles Chen, are also attempting to extend various phones' ability to read back scanned text to include signs that are anywhere in the phone's field of view.

Reading Signs

You could probably translate different things about the text into different audio cues. Bigger text? Louder reading. Stereo headphones could also make the reading come from the direction of the sign. As image recognition gets faster and more accurate, I'm sure a number of different audio cues could tip a blind user off with clues about the environment. I am sure the usefulness of this will go well beyond blind users.

Re:

Or we could admit that the current crop of iPhone inspired phones are a tremendous leap backwards. Seriously, even if you're not blind, the fact that you have to pull it out of your pocket to use is a pain in the ass. Makes me wonder what the point of wireless headsets really are if you ultimately still have to look at the phone to use it.

I'm not sure any of that other stuff is going to be helpful if companies are exercising in a pathological hatred of necessary buttons.

Re:Reading Signs

>Seriously, even if you're not blind, the fact that you have to pull it out of your pocket to use is a pain in the ass.

Yeah, because technology such as 'hands-free Bluetooth' hasn't been invented yet on your planet.

When I'm using an earwig or in the car, the phone stays in the pocket...everything, from connecting to the car to taking calls happens automatically or via voice control - what a country.

Re:

Then there's also the research going into piping images from cameras directly through the optic nerve. Wont work for everyone but it's something.

Re:

Bigger text? Louder reading.
ALL CAPS FOR SHOUTING!
obviously.

They should follow the Screenless MP3 Player.

I wish I could remember the name of this device, but it was essentially a MP3 player with no screen; just directional buttons and voice-based navigation. It was manufactured for blind users, and it worked wonderfully. It could even read text files and accept CF cards for expansion!

I think that a cell phone with just buttons on it and braille lettering would suffice, provided that the voice navigation is really good. An added advantage is that having no screen can make for very thin and attractive devices, if aesthetics is something of a priority for them.

Re:They should follow the Screenless MP3 Player.

One of the down sides to hardware that has no screen on it is that there is not a sufficient market for general use, and as a result the cost to the consumer is significantly higher. Braile readers for example end up being hand made because there isn't a sufficient demand to mass produce them, but the result is a display that's one or possibly 2 lines of text, that costs a couple hundred to a couple thousand dollars to produce.

Part of the idea of what Raman is doing is taking existing consumer hardware and applying software solutions to make them more usable to the blind.

A similar solution would be to use a cell phone to convert the words a person is speaking into text that a deaf person could read. You could build into it language recognition and translation and the deaf person on the bus next to you on your next tour of Europe, Japan, China, etc. may get more out of what is going on around you than you do.

A screen free MP3 player may sound like something that a blind user would appreciate, especially if the control hardware, and the audio menus were well designed. The big problem becomes one of what all the device can do. Adding a camera to a device like this seems somewhat counter intuitive from a consumer goods perspective, even if the hardware cost were just additional pennies. How about gps location hardware. Again just pennies, but to build it into that MP3 player doesn't seem to make all that much sense, does it. Add in a compass and an accelerometer, and you have a navigation aid that a blind user could use to get almost anywhere with.

The thing is that pretty much everything described is in a G1, so really adding usability for the blind is primarily a matter of plugging into the user interface with something that converts text to speech and where possible images to text. Both of those are partially solved problems. The part that's still difficult is getting it all tied together, and making available sufficient processing power to make it operate cleanly.

Brilliant

"Since he cannot precisely hit a button on a touch screen, Mr. Raman created a dialer that works based on relative positions. It interprets any place where he first touches the screen as a 5, the center of a regular telephone dial pad. To dial any other number, he simply slides his finger in its direction -- up and to the left for 1, down and to the right for 9..."

So simple yet so brilliant. There is so much tripe published about 'innovation' (usually Microsoft), yet I think this is the first time _I_ can use this word properly.

Well done Mr. Raman - truly brilliant.

Re:

I was coming here to say pretty much the same thing - it's a great and simple idea that would even be of use to people with full sight. I often have problems 'dialling' a number on the touch screen of my HTC TYTN II - 'fat fingers' and all that.

Re:

This isn't innovative - see mouse gestures, pie menus, even context menus that pop up where you first click, and not in a predefined place.

Now if this guy Raman had instead invented Raman Noodles ...

Re:Brilliant

Maybe true - but does any touchscreen phone currently on the marker have this damn useful feature?

Re:

All regular touchscreens (not phones) do. You can configure them so that a "double-tap" opens, for example, the context menu right under your finger, or launches a specific app with a specific interface at your fingers' location.

Calling this innovative is like calling a blank white board innovative because "you can write anywhere - you're not constrained by the lines."

In other words, this is not an "invention" - it fails the "obviousness" test.

Re:

This is no more different than popping up a window at an arbitrary place on a touch-screen. In other words, it's over a decade old.

Re:

Is the whole thing really brilliant, or is what Mr. Raman doing just an elaborate work-around for what was essentially a usability step backwards? One thing that I like about my Treo phone is that it has a combination of a touch screen AND has real buttons that I can feel with my fingers and not have to look at the screen if I don't want to. Hell, I can call into my office phone system, put in my voicemail authentication codes, and listen to my voicemails without ever having to look at the phone...

Re:Brilliant

How a bout having a set of predefined buttons with a small bump so that the number 5 can easily be identified. The buttons could be placed in some sort of logical order, let's say in ascending numbers, 3 each row. We could also add * # Send and End as buttons.

It might look something like that:
S E
1 2 3
4 5 6
7 8 9
* 0 #

What would really be cool and useful too is to make sure that when someone presses a button it also feels like its been pressed.

As an aside, a cool feature would be the cellphone speaker phone saying the number that was pressed as someone dialed. _That_ would be useful.

what about speech recognition

afaik there are plenty of phones on the market that have speech recognition such as speaking 'mum' and it will auto dial, ive never used them so i dont know what sort of quality they are at atm but that seems to me like it would be a better way of doing things, since it also means you wouldnt need to memorise every number you might need

are there any blind /.-ers whom might be more qualified to give an opinion?

I Wonder

I suspect that this technology will be used more by drivers and teens than the handicapped.

Re:

Of course it would. A phone with keys with braille (or raised numbers/letters or whatever) on them would be better for a blind person - why would they want ab LCD screen covering the whole device even if it is a touch screen?

The article does mention that, something along the lines of the guy likes to think about things in terms or what if the user can't look at the screen right now, rather than what if the user was blind.

Why not..

If we're working on making a touch screen phone more accessible to the sightless, why not ditch the screen entirely and replace it with a tactile display capable of adapting to the needs of the user? This would make it possible to still have your email or even text messages right at your fingertips - literally!

Here's a prototype that I'm sure could be improved upon and made portable given the right amount of funding.

http://www.nist.gov/public_affairs/factsheet/visualdisplay.htm [nist.gov]

Problems

There are some problems with the method he's chosen. Most notably 7,9,1,3. It's pretty hard to configure the bias for diagonals so that people don't accidentally do a diagonal instead of a direction or vice versa. It may seem easy to do but when you're dealing with an inch of travel, a slight amount to the left or right can make a huge sifference to the angle. There are also issues with phone alignment in the hand.

I think Ultimately the best solution is embossed numbers on the phone or other tactile metho

Eyes Free ??oneoneone?

Where will this end? First they'll make a cellphone that has no eyes and the next thing you know they'll make one with no spleen.

I better hold onto my Nokia SpleenMaster 5000, it might be the last of its kind.

Re:

+1, Relevant Sig

That's pretty cool

I went to a college that catered heavily to people with disabilities, primarily the deaf but also a large number of the blind. It occurred to me early on that a great deal of the tech they use is developed by people without disabilities and then tested on people with them.

I think Mr. Raman is pretty unique in the sense he's able to develop like this as a blind man. That said, I'm curious why voice recognition wasn't considered the better option?

SWYPE

What's the difference between this and swype [slashdot.org]?

Re:

swype is intended as a replacement for an entire keyboard, and presumes that you are reasonably close to the first letter in the word.

One way to make swype work like this would be to start with a 'common' character on the keyboard and draw from there. Preferably central to the keyboard. Another alternative would be to build a 5x5 grid of letters, doubling the c and k characters into one cell of the grid, and using that as a swype board. (It would fit on a screen better as well) That layout is what the tap c