Robot Swarm Control On Microsoft's Surface

zerOnIne writes "Dr. Mark Micire of UMass Lowell has built an intriguing new user interface on the Microsoft Surface, a multitouch-capable table computer. The interface is being used to control swarms of robots for disaster response, search, and rescue. One of the most interesting things about it is the intuitive tabletop joystick widget. Using a very fast hand-detection-and-identification algorithm, they can paint a touch joystick (dubbed the DREAM controller) directly underneath the hand. This joystick conforms to the size of the user's hand and tracks with hand movements, making sure that the control is always directly under the hand where the user expects it, even without haptic feedback. I've had a chance to go hands-on with this system, and I think it's truly remarkable."

Re:RTS games?

[SharpFang]

The problem with RTS games path finding vs robot path finding is that the games have a flawless description of the environment (the description IS the environment) and no inherent physical limitations on the driven device, only ones programmed in.

A real obstacle can move, can be hard to spot or misrecognized as non-obstacle, can resist traditional methods of surmounting it (say, is slippery or crumbles). A real robot has to deal with traction slipping, in route deviating due to slipping on the surface, limited acceleration and braking power, environment behaving against specs (tell a crumbling building to follow the computational model...), communication shortages and so on.

Also, this is a demo, to let people see how that works. I believe it could be done 20 times faster by an experienced operator doing actual work instead of a demo.

Re:Is it just me?

[herks]

The swarm stuff is neat, but has been done in RTS games. Nothing really revolutionary, except that it's never been done for swarms, using a touch table. The work defines some easy gestures and methods that have been studied and found to be what the majority of people would naturally do. What I find truly revolutionary about this interface is the dream controller. It doesn't get much play in the video, but it is remarkable. You put your hand down, and it draws the controller around it. It's exact every time. No matter what size hand, or orientation, it draws it perfectly for 100% of the human population. No more xbox/ps3 controllers that by their physical nature can only be built to be comfortable for the average. (50% of the population.) Having the tactile feedback of a keyboard or gamepad button has it's benefits, but if the controller is drawn directly under your hand, You can very quickly learn to press a button without having to feel it. Having this possibility opens up a whole new world of controller design with applications in video games, touch screens apps, every day appliances, and more. We can rethink controller design, and make the controller conform to the user, rather than the user conforming to the controller. I think it has a lot of potential, and am looking forward to seeing where this goes.

Re:Unbelievable

[cptdondo]

I've done disaster response. While robots may have some very limited applications, dogs and eyeballs and boots are better. You need lots of them and each one has to be thinking and looking on their own.

You are looking for survivors and making decisions that are not quantifiable at all. It'll be a long time before we can send out a "swarm" of robots into a disaster area and have them work as well as a trained dog or a trained SAR person.

Real robots too!!!

[Anonymous Coward]

Check out his other videos. They have done this with real robots also.... http://www.youtube.com/watch?v=48A8vdJ68lI

Re:Unbelievable

[Hunter0000]

The swarm of ground robots is a bit of a reach goal. Current applications of this include showing manpower and equipment (and current robotic resources, like UAVs). The idea being, issued orders to a person or vehicle's representation on the map will send orders to a communication device they posses. The map would also serve to display data that currently is compiled (slowly) onto paper maps. This way, the interface becomes useful for combining data and asset display that also allows orders to be issued to those assets.

(I work next door)

Controlling Home Robots from "Smart Surface"

[HizookRobotics]

In case you're interested, there has been some nice work in using tabletop interfaces (ie. Microsoft Surface) to control home robots [hizook.com], like the iRobot Roomba.

Re:RTS games?

[SnowZero]

This demo was a simulation, so there really isn't that much difference from a good RTS. Motion control and local path planning are well understood.

While the demo looks cool, it is not really much different than the robot GUI I wrote for my robocup team that could control 10+ robots (real or simulated). It used a mouse and any number of ps3-style controllers. And yes, I got my ideas from RTS games, and some other teams had even better GUIs.

Also, it would have been nice if they didn't speed up the video, so we could better understand how well it worked. One thing I often find in touch interfaces is that selection is slower than you'd expect since you block the item with your finger. A mouse pointer is much smaller, and works well for skilled operators.

Re:VIRTUAL robot swarm control on MS surface

[zerOnIne]

Not in this video, but they have controlled real robots, too:

http://www.youtube.com/watch?v=48A8vdJ68lI [youtube.com]