Designing a Practical UI For a Gesture-Based Interface 44
An anonymous reader writes with a link to an intriguing account of the challenge of designing a close-range, hand and finger-based gesture recognition interface using 3D cameras. Things like this look good in science-fiction, but it's hard to create a gesture-based system that makes sense to the user and rejects gestures not meant for the computer.
Test it on Italians (Score:5, Funny)
You're going to have to test it on Italians for gesture rejection. I can't imagine what kind of havoc could be created if the interface over-saw even the most mundane of conversations.
Re: (Score:2)
You're going to have to test it on Italians for gesture rejection. I can't imagine what kind of havoc could be created if the interface over-saw even the most mundane of conversations.
We already know where this is going to lead:
For years radios had been operated by means of pressing buttons and turning dials; then as the technology became more sophisticated the controls were made touch-sensitive - you merely had to brush the panels with your fingers; now all you had to do was wave your hand in the general direction of the components and hope. It saved a lot of muscular expenditure of course, but meant that you had to sit infuriatingly still if you wanted to keep listening to the same programme.
If Microsoft is involved... (Score:3)
Re: (Score:3)
American Sign Language? Oh wait you mean something else.
Seriously whats wrong with ASL? Large installed base, lots of training, could be pretty darn useful as a translator when someone without ASL "talks" to someone with ASL.
My guess is MS would try to embrace extend and extinguish ASL... make it just like ASL but upside down and backwards, just to make it hard for everyone.
Re: (Score:2)
If you mean the finger then it is not as widely used or recognised outside the USA ... ...many gestures are very localised and do not travel very well to new cultures
Re: (Score:2)
It wasn't. Due to the international success of US-made entertainment media, their expressions and gestures have been spreading for some time. The finger is certainly recognized here in the UK.
Take a look at the Leap (Score:3)
more challenging...practicality. (Score:3)
The biggest challenge is explaining who/how/why anyone would NEED a gesture-based interface.
Sure, we all thought Minority Report looked cool with the flying window-thingy, but honestly, Mr Cruise could have 'flipped through' his data far, far more easily sitting at a desk with trivial motions of a mouse instead of giant arm-wavings and calisthenics.
Plus, then he'd get to SIT, which is a little more conducive to surviving the 10+ hour days most of us spend staring at a little computer screen.
Re: (Score:2)
Re: (Score:3)
I dislike gesture UIs. Not just because they are new. The main reason is that if you are a newbie there are no hints about what to do. With shortcut keys you could always use the menus instead if you don't know the keystrokes. But Windows, Blackberry, Android, iPhone, all have different gestures its easy to get lost if you use more than one platform.
Re: (Score:2)
Plus, then he'd get to SIT, which is a little more conducive to surviving the 10+ hour days most of us spend staring at a little computer screen.
This is actually a minus, and a big one at that. Standing up desk is a LOT healthier.
Re: (Score:2)
Agreed. The mousewheel has become ubiquitous because it out-competed the effort required to twitch the mouse to move the pointer over to the scroll bar.
Aside: Have any comedies spoofed gesture interfaces? It seems like it would be a good target for a physical comedian (Rowan Atkinson, Lee Evans or Ryan Stiles for example).
As an ASL Interpreter... (Score:5, Informative)
I interpreted ASL in educational settings (High School, Freelance, University, Public, and even elementary.) for something like 6 or 7 years.
My arms were ripped, and you could expect to burn several hundred calories (EASY) during a day of doing that. Also, I had learned the stretch properly thanks to some Aikido training.. and I still had some bad habits that caused me repetitive stress problems.
Gestures are a novelty, and a lot of work for the user... I think there will be many blind alleys before they become natural.
Some problems/ideas I see:
1. Exhaustion - you waste a lot of energy
2. "Namespaces" - you can make two gestures at once - geez... so you have a left hand gesture that tells the computer to listen (the ASL "Attention" one handed would work) + a command - maybe even "against" that hand. Its like a salute with your left hand vertical moving away from your face.
3. Facial expressions are a HUGE part of ASL, probably not even considered. "WH" questions get eyebrows scrunched, other queries eyebrows up, puffed cheeks and all kinds of things...
4. Security - I defy you to sign EXACTLY like someone else... It's possible, and easy in a mocking sense (High schoolers) - but I imagine a door that could see you carrying groceries and unlock combined with voice recog., or other simple things would be useful.
Re: (Score:2)
Well.. ok.. attention would be both hands... but I meant it in context of using it to communicate with a computer.
Re: (Score:2)
All of your problems exist only because you are trying to use the wrong tool for the job.
The HUGE advantage this would have is that it could be an avenue for teaching people how to move their bodies properly. RSI happens when you don't know how to move proper
Re: (Score:2)
Don't use ASL it is unlike English specific to USA/Canada and West Africa is not mutually intelligible with most other sign languages and is largely unknown outside the Deaf community
There is no deaf sign language that is reasonably universal like English is in spoken languages so using ASL as a base will alienate many of your potential users ...
The biggest issue is "Gorilla Arm" if you are making a lot of small precise gestures with arms out you get tied extremely quickly, which is why most sign languages
Too bad, so sad. (Score:1)
How do you do it when dreaming? (Score:2)
Re: (Score:2)
Re: (Score:2)
Aah, young Jmc, do you not recognize that Matrix and Star Wars are two different things?
There is no force. No matrix either.
Re: (Score:2)
Never heard of metaphors?
Any one who has enough control in their dreams to do what the parent can would understand. Perhaps you need to educate yourself in lucid dreaming?
Re: (Score:2)
I've done t before, but it's not as interesting as uncontrolled dreaming. I've also done multi-path dreaming: multiple dreams at the same time; but only once. I've also dreamed specifically in color once or twice.
Re: (Score:2)
Lucidity at night is directly correlated to awareness during the day.
What about intent recognition (Score:2)
Optional (Score:2)
I think there is a good solution to this problem (Score:1)
It's the solution humans use in gesture-based communication. Eye contact. The computer needs an obvious eye, and the software needs to be able to tell when the user is looking at it. Bonus points if the eye can blink or move or otherwise express the computer's understanding of the gesture.
Hitchhiker's Guide to the Galaxy. (Score:2)
Have to quote this from _The Hitchhiker's Guide to the Galaxy_ :
A loud clatter of gunk music flooded through the Heart of Gold cabin as Zaphod searched the sub-etha radio wave bands for news of himself. The machine was rather difficult to operate. For years radios had been operated by means of pressing buttons and turning dials; then as the technology became more sophisticated the controls were made touch-sensitive--you merely had to brush the panels with your fingers; now all you had to do was wave your hand in the general direction of the components and hope. It saved a lot of muscular expenditure, of course, but meant that you had to sit infuriatingly still if you wanted to keep listening to the same program.
The fundamental problem of gesture recognition has been recognized for quite some time now.