Please create an account to participate in the Slashdot moderation system

 


Forgot your password?
Close
typodupeerror
×
Technology

GUIs for Robots 156

Posted by michael from the click-drag-boom dept.
kabir writes "OpenGL.org has a link to a fun article over at the Stanford Aerospace Robotics Laboratory. It seems an OpenGL-based GUI has been developed to enable the operation of multiple complex field robots by a single operator. The interaction mechanism was inspired by interface techniques refined in the Real-Time Strategy (RTS) genre of video games. Fifty years from now I expect all wars to be fought by giant robots controlled by teenagers."
This discussion has been archived. No new comments can be posted.

GUIs for Robots More

GUIs for Robots

Comments Filter:
  • GUI Development

    A Graphical User Interface (GUI) has been developed to enable the operation of multiple complex field robots. The interaction mechanism was inspired by interface techniques refined in the Real-Time Strategy (RTS) genre of video games that includes the popular titles Starcraft, Command & Conquer, and Strifeshadow. This mechanism follows three basic steps:
    The operator selects which robots to use
    The operator selects which objects to be acted on
    The operator selects a task to perform
    However, the nature of field robotics requires some significant differences in the implementation of the RTS interface method. For instance, there is no single source of accurate global information -- each robot can only provide relative data that has to be fused together. In addition, the tasks that each robot can perform change dynamically and this information must be reflected in the choices presented by the GUI to the operator.

    The GUI utilizes OpenGL to display the robot world in three dimensions. Development was significantly aided by Glt (by Nigel Stewart) and GLUI (by Paul Rademacher). Using Glt, which includes GLUI, is highly recommended, especially for C++ programmers new to OpenGL. The OpenGL picking mechanism was used in conjunction with GLUI dialog boxes to provide a direct manipulation interface for robot operation. Additional screenshots and system architecture diagrams are also available.

    In the background, real-time data is being handled by NDDS from RTI. The determination of robot capabilities, which change from moment to moment depending on robot capabilities and object characteristics, is performed by the Java Theorem Prover (JTP) developed at Stanford by Gleb Frank. Communication between the GUI and JTP is carried out by the Open Agent Architecture (OAA) from SRI.

    SWAT Observations

    To gain insight into how humans already manage distributed teams, this research observed field exercises of a police Special Weapons and Tactics (SWAT) team. The Palo Alto / Mountain View (California) Regional SWAT team provided access to its training exercises. The researchers were given free movement throughout the exercise area so that the activities of the commanders, the field units, the snipers, and the hostages and suspects could all be monitored. The tactical commander and field units play roles analgous to the robot operator and the field robots, respectively. The key observations made were:

    -The role of the leader (commander or operator) has two primary components
    *Cultivating common ground
    *Coordinating action

    -A natural and efficient interaction can be based on physical objects in the remote agents' (field units or robots) environment, just as with the RTS games
    • I did the same thing for a little 3D platform I made (which I want(ed) to mount an airgun on).

      It was parport controlled and I used OpenGL to draw a crappy picture of it onscreen in a Qt application =D
    • There were two interesting stories that come to me. One is Enders game, where teenagers do remotely control the fate of the world. The other series eludes my brain right now. I guess its not late enough at night for it to work. This story basically has the same idea, remote operators control robots in war. The main thrust of this second story is that after a while, the military had found that having the controllers close on the ground was more effective in close quarters combat. The controllers were described as regular (age wise) people. Thus this "new" group of combat soldier was able to beat all others consistently with the same technology.

      One question raised, will teenagers/preteen be needed to run this or will "older" people be just fine. The second series talks about the feel of battle and how camera's and tech senses does not quite match what a person can feel about the battle. This raises the question, can a battle really be effectively fought remotely. I suspect we cannot answer these questions until the technology appears and are tried.

      On a paranoid note, it is a nice thought we can bring a battle to others without cost to our own troops lives, but when (not if) every one can do that, then civilians will become the target more and more. This effectively makes everyone a military target. End of paranoia (for now :).
    • (Score:1, Redundant)

      How's this redundant if it's the first post? Heh...

  • Games (Score:5, Insightful)

    by GigsVT ( 208848 ) on Wednesday June 26, 2002 @08:13PM (#3775328) Journal
    The interaction mechanism was inspired by interface techniques refined in the Real-Time Strategy (RTS) genre of video games.

    This is interesting. Often it seems the games are ahead of the "serious apps".

    One of my friends in high school put together a level in Duke Nukem 3D that was based on our high school. It was pretty accurate. This was pre-columbine, so nobody was thrown in jail for it.

    If these game companies just put a little extra effort into retrofitting their game engines for serious purposes, they might sell the same thing for hundreds of dollars a copy. Imagine something like a first person shooter combined with The Sims. You could design your whole house, to scale, and then walk through it.

    Click a menu option and output a postscript file to send to the structural engineer and architect for tweaks and approval.

    Whoever does this will probably make lots of money.
    • You could design your whole house, to scale, and then walk through it

      It has already been tried with the unreal engine back in 1998.

      Check out this site..
      http://www.unrealty.net/
      They are considering trying again with newer technology.
    • Well, something related has already been designed.

      The article [bbc.co.uk]

      While you can't actually design the buildings within the modified engine, you can walk around in them first person.
    • Look at Paul Rajlich's [uiuc.edu] VR model of his home for his CAVE here [uiuc.edu].

      In fact, it's not very tough designing something like that. Anybody who's tried designing games or game levels, even simple ones would know how easy this is.

      For example, look at NeHe's [gamedev.net] simple 3d engine demo [gamedev.net] - you could easily build any structure you want for one of these, with sufficient skill modeling your house wouldn't be any more difficult than mapping a few co-ordinates. It'd be cooler still if you could import some format like DXF or VRML or the like into a suitable map.
    • One of my friends in high school put together a level in Duke Nukem 3D that was based on our high school. It was pretty accurate. This was pre-columbine, so nobody was thrown in jail for it.

      I wonder if it'd be okay to do that now as long as you were "rescuing your school from terrorists." :-) (Just don't make the terrorists look like anyone in the administration or the football team.)

  • Japanese sci-fi predictions? (Score:5, Funny)

    by Cryptnotic ( 154382 ) on Wednesday June 26, 2002 @08:16PM (#3775357)
    Fifty years from now I expect all wars to be fought by giant robots controlled by teenagers.

    That's what Japanese science fiction aniume has been predicting for at least 23 years. Macross, Gundam, etc...

    • Doesn't Japanese anime predict that people will actually be driving/flyign these giant robots, not controlling them from hardened bunkers?
      • Having the pilot inside the vehicle means that the enemy cannot jam the controls as easy. This leads to problems with the idea of teenagers driving these. The g-forces of movement and other movement realted issues are always a problem but the military puts a lot of effort into training their pilots about such issues. A real pilot must be VERY healthy to fly in a combat situation...
        • Technicaly speaking, you are at your healthiest in your later teen years. Assuming of course you keep a regular exercise regimine.
          • that is half true...

            during your late teen years your body will use oxygen more efficiently and your muscles will require less energy to work... overall your body is more efficient but you have not had the time to build a large muscle mass or develop an all around endurance for your body...

          • Re:Japanese sci-fi predictions? (Score:1, Funny)

            by Anonymous Coward
            If my healthiest years are in my late teens then i'm f*cked ... looks like everything is down here from here..
      • Yes, for the most part older anime predict combatants riding in the vehicles they are controlling. However, there is one recent show called "Starship Girl Yamamoto Yohko" which predicts remote-controlled high-tech combatants (not giant robots exactly, but more generic fighting spaceships). In that program, cute girls who are also avid gamers (a novel idea) are recruited to pilot these ships in remote-controlled combat for the purpose of resolving disputes.

        • Watch it again--Yohko and the gang really are inside the battleships.
          The key is that they're using teleportation-based ejection systems, so if they take a lethal hit, the pilot is returned to the command center without a scratch.
          • Damn, you're right. I only watched the first few episodes of the TV series (not the OAV) in Japanese with no subtitles. I guess I misunderstood.

      • Some of them have predicted a combination of the two, which I feel is the most likely prediction. A single, highly trained commander pilots a robot and is then assisted by several somewhat AI controlled robots that obey his commands. Thus, the robots cannot go out of control because they require commands and the signal can't be jammed as easily because of the strength of the signal due to the short distance, but the entire platoon of robots still act with one mind.
  • 1) Don't take your eyes off the screen. You might get fragged.

    2) Don't take your hand off the mouse. You might get fragged.

    3) You only have one life. Use it wisely. And don't get fragged.

    Hlynna
  • I wanna play too

  • The future of productivity.. (Score:4, Funny)

    by NanoGator ( 522640 ) on Wednesday June 26, 2002 @08:20PM (#3775382) Homepage Journal
    "interaction mechanism was inspired by interface techniques refined in the Real-Time Strategy (RTS) genre of video games."

    I was joking around one day PHB style bosses will want to use an RTS interface in Office to control their employees. That way, they can avoid having to actually talk to them. I stopped laughing when I realized that the technology to do that is almost in place. All we need is wirelessly connected PDA's....

  • Fun with Robot Combat, Today! (Score:4, Informative)

    by toupsie ( 88295 ) on Wednesday June 26, 2002 @08:21PM (#3775391) Homepage
    Visit IBM's Robocode [ibm.com] and program your own deathbot! If you are old school Mac, you might remember RoboWar [aol.com]. A favorite of mine when I had a PowerBook 140.
  • This is a little off-topic, but speaking of RTS interfaces ...

    It seems to me that the next major leap in RTS games will come with voice control. This article encapsulates it nicely: you choose who you want to act, you tell them what action to perform, and you tell them where to perform it. In the specific application of, say, starcraft, I envision it being something like this:

    Nexus Build Probe
    Probes-On-Screen Mine Ore
    Zealots-On-Screen Form Squad-1
    Squad-1 Move Here [Mouse pointer]
    Squad-1 Attack Firebats

    There would be lots of detail to work out, and probably some fuzzy logic about which target is meant by firebats for example, and there's the problem of specifying certain areas (does 'here' work?) but I think once it was working it would provide a much more fluid interface with the game. This can't be far off, right? Hell, my cellphone already responds to "Call Batman On The Mobile Phone"
  • Here's another way to control a robot with a GUI: RobotFlow [sourceforge.net]
  • "Fifty years from now I expect all wars to be fought by giant robots controlled by teenagers."

    Yeah...but will they "shout because their weapons are voice activated"?

  • All they need to do is install quake and benchmark it. That or use quake as the intereface to "use" the robots... which in reality just means make them kill things.

  • Voice activated (Score:1, Interesting)

    by dpavlenkov ( 175903 )
    Why voice? put a retina tracker on your eye, blink to lock on target, blink again to unlock, and blink twice to destroy. You can use your voice to do other useless things in the meanwhile.
    • Blinking doesn't quite work. The human eye naturally blinks very frequently. Easier way to do this is just to have normal buttons. I know it's not glamorous, but it works.
      • Eye tracking could still be usefull for aiming. Look then push the trigger. IMHO a combination of voice, eye, hand, and foot operated controlls would be best.

        • That was pretty much my point. Just because some interface is new and wonderful, does not mean it's suited for every task. Eye tracking is perfect for aiming. Finger triggers for firing, (or maybe some kind of touch screen). Foot pedals for steering. Any kind of "five seconds ago" things would be for voice control, ie "chaff" or "decoy".

          Obviously, a lot of thought would have to go into interface design, and I probably missed quite a few things. Probably the best way to figure out what goes where would be to have a simulator where you can define bindings, and then track what bindings the most successful people come up with, and base your work on those. Of course, factoring in bias from previous experience in piloting of any kind.

  • Robots (Score:1, Funny)

    by 0vi_king ( 514106 )
    % The Commandant addresses the graduating class.

    The wars of the future will not be fought on the battlefield or at sea. They will be fought in space, or possibly on top of a very tall
    mountain. In either case, most of the actual fighting will be done by small robots. And as you go forth today remember always your duty is
    clear: To build and maintain those robots. Thank you.

    -- Military school Commandant's graduation address, "The Secret War of
    Lisa Simpson"

    http://www.snpp.com/episodes/4F21

  • OB Simpsons (Score:1, Redundant)

    by ocie ( 6659 )
    Your duty is clear. To build and maintain those robots.

  • Questions to developers... (Score:4, Interesting)

    by josh crawley ( 537561 ) on Wednesday June 26, 2002 @08:38PM (#3775546)
    Article said:
    ---"However, the nature of field robotics requires some significant differences in the implementation of the RTS interface method. For instance, there is no single source of accurate global information -- each robot can only provide relative data that has to be fused together. In addition, the tasks that each robot can perform change dynamically and this information must be reflected in the choices presented by the GUI to the operator"

    Well, umm. Yes there is a single source of global information. It's called GPS. I believe, using the correct civillian gear, you can get centimeter-accurate location points. Also, political/geographic maps are available from many locations from different governments.

    The best way I see that this can be used is that you create a neural network between the robots by way of bluetooth. The robots could share GPS location data in realtime. Make sure that they can pass on data to out-of-range units.

    Once you have location links, you can piece screenshots (by using angles of the GPS coords). Personally, I'd craft chips designed specifically for piecing the screenshots to a 3d locale. No sense having a generic chip (x86 or whatnot) doing that. And NO beowulf clusters :-)

    • Wrong global (Score:3, Informative)

      by wowbagger ( 69688 )
      You misinterpreted the meaning of "global" - they did not mean co-ordinate data, rather they meant the state of "the world".

      In a game, the gameserver knows where everything is. In robotics, the control program doesn't know for certain that the blue cube is at 0.1x3.5y99.1z - it has to get that information by looking at what the sensors on the robots say, and those sensors lie. So the control program has to take all the data from all the robots and try to fuse it into something meaningful, all the while keeping in mind that "things are not what they seem".

      That is why controlling a real-world robot is MUCH harder than controlling a player in a video game.
      • I have to agree with wowbanger as I am a Controls Engineer. One of the things I do is program industrial robots.

        Most all of these are single arm robots (not the walking around things that you see on TV) with either 4 or 6 axi of motion. The accurracy is .001mm and you are talking about using a GPS? I don't think so.

        • That's WOWBAGGER, you bag-licking kneebiter ;)

          I've done that sort of work myself, so I know whereof I speak as well (what I always hated was when the damn steppers would cog, and I'd lose track of where I was - I didn't have the luxury of an encoder back then...)

          But the more important part of this isn't just knowing where the robot is, but knowing where the NON-ROBOT objects are - sure, if we use IPv6 and assign every object in the universe an IP address and position tracking, we could solve that. But I think that would get a little expensive....
          • Wow! I am a bag-licking knee-biter!

            Thanks!

            Damn... steppers? Servos rule man... especially when you use resovlers instead of standard encoders.

            I didn't intentionally mangle you nick. I wrote that reply before properly caffeinated.

            • No offense taken - I figured as much.

              Also, you do realize the above insult is a "freebe" - given out of order. I'm still working through the <V'hhrg character not available in your charset>'s, so I won't be getting to you soon.

              As for steppers - hey, when you are controlling twenty of them with a single Z80 and no hardware assistance, you take what you can get.
    • Correct civilian gear for centimeter accuracy costs at least $100,000. At the low end. And then it's really a pain to use.
    • "Well, umm. Yes there is a single source of global information. It's called GPS."

      What if your robots are underground? No GPS there...

      (I work for a company doing underground robotics)
    • It sounds so easy, but localization in mobile robots is actually a very difficult problem. GPS is great for some applications (for example, helicopter robots [usc.edu] but (of course) it doesn't work indoors and it doesn't work well at all in built-up areas (due to lack of line-of-sight and multipath problems - just like your cellphone).

      One of the main potential military applications of robots is working in built-up areas, because these are so hazardous for soldiers. DARPA sponsors a LOT of work in this area, for example the MARS program [darpa.mil].

      The current most successful approaches are all broadly statistical, providing a means to "see through" the noise, drift and variations in robot sensor readings. Sebatian Thrun [cmu.edu]'s group at CMU has some of the best work in this area (for an overview, see this review paper [cmu.edu]. Andrew Howard [usc.edu] at USC has some cool movies here [usc.edu] showing his technique based on a physical spring/damper metaphor. Great stuff.

      This problem is here to stay. If you have ideas, join a grad school program and help out!

    • Global information is not just position information, it's world state information.

      Fire fighting robots - for example - might be the only way for which smoke level and heat level info at there location is communicated, but there is no single known state (or Global source) for this information. And it changes with time. Robot 5 reports it can no longer advance due to a heat threshold level hit, not the GUI operator has one less function availible for that robot (or perhaps more.)
  • Yeah, teenagers controlling battles. Someone reads a little too much Orson Scott Card maybe...

  • Fifty years from now I expect all wars to be fought by giant robots controlled by teenagers.

    Which is a lot better than having our teenagers fighting in person !!!
  • heck this was rejected as uninteresting months ago

    2002-02-12 14:54:27 Operating Systems for Robots (articles,programming) (rejected)

    being summer, maybe folks have time to read now.

    ;-)

  • Sweet, now I can finally play Starcraft in real time, not on some computer in stupid simulated time... what will the think of next???
  • Imagine the carnage if you combined this [slashdot.org] with this [sourceforge.net].

    :-)

  • wars fought by robots in teh future controlled by teenagers. hmmmmm is anyone here reminded of enders game. just wondering.
    • hmmmmm is anyone here reminded of enders game. just wondering

      Yep, I sure was. Ender's Game by Orson Scott Card. In this book children are trained in an orbital military academy. They are trained in tactics in a weightless environment. Great book. I enjoyed how it explored the intricacies of "schoolyard politics." Highly recommended (by me), but the rest of the Ender series was so-so IMHO.

      BTM
  • Though it's not a particularly good book, the first (and best) part of Joe Haldeman's Forever Peace features tele-operated U.S. combat robots in Central America being controlled by "Mechanics" back at a high tech base.

    After that, alas, the book turns into an Idiot Plot with sadistic fundamentalists trying to blow up the world, the prevention of which requires that liberals have to peform brain surgery to mentally link everyone together and make them think right.

  • Enders Game. Orson Scott Card. Nuff said...
  • "d00d, ur hax0ri|\|g 4g4i|\|!"

    "I'm not camping you camper fag!"

    "I 0wnz j00!"

    etc...
  • Stanford has several robotics labs, of which this is the best.

    GLUI [unc.edu], though. Yech. That's a crappy toolkit. You have to modify the guts of the thing to add new widgets, the architecture is a mess, and it has problems synchronizing the front and back buffers. GLOW [caltech.edu] is much better. I've used both. Both are menu and widget toolkits built entirely on top of OpenGL. This gives cross-platform portability. Doing 2D widgets through the 3D OpenGL engine seems inefficient, but it works well. If you have 3D hardware, you may as well use it. It's an relatively clean way to program.

  • "Fifty years from now I expect all wars to be fought by giant robots controlled by teenagers." Voltron! Power Rangers! Tranzor Z! Cool Coooool!!
  • the contrys that have not the money to buy such robots? Would there be robots trampeling through these contrys just steping on ppl?
  • What if someone hax0rs the robots, like the enemy?
    Seriously.
  • Fifty years from now I expect all wars to be fought by giant robots controlled by teenagers.

    Predator, Global Hawwk, Dark Star anyone?

    But realistically, the 3rd world will still be the 3rd world, even 50 years from now. And will still be fighting over the same patches of dry barren ground as they are today.

    It doesn't count as a win until some 19 year old with a rifle is standing on that patch of ground
  • I'm not the only one who thinks we may one day have wars in which robotic troops, equipment, ships, etc. are controlled remotely by kids. Orson Scott Card thinks so too. And anyone who plays RTS games online will tell you that, while most players have the strategic ability of an antique doorknob, there are a select few who are absolutely amazing.

    In Ender's Game, OSC wrote about children commanding fleets of starships in interstellar war, while thinking they were merely playing a game. This type of scenario is looking more and more plausible every day.

  • wars fifty years from now (Score:3, Insightful)

    by danox ( 232017 ) on Wednesday June 26, 2002 @10:44PM (#3776367) Homepage Journal
    Fifty years from now I expect all wars to be fought by giant robots controlled by teenagers

    What I think is more likely is that fifty years from now we will see the US decimate more 3rd world countries using robots controlled by teenager, who only think they are playing a game and thus will never reveal what they have done to the media, and in fact no one will ever find out since the press were never informed, and those who try to find out will be detained indefintately without trial for allegedly breaking the latest anti-terrorist laws which prohibit attempting to criticise national defence actions in any circumstances.

    Well come to the wonderful future Mr. Bush is building for the world.

  • How will they deal with map hacks?

  • A total reversal (Score:5, Insightful)

    by MarkusQ ( 450076 ) on Wednesday June 26, 2002 @11:23PM (#3776591) Journal

    Fifty years from now I expect all wars to be fought by giant robots controlled by teenagers

    In stark contrast to the situation today, where wars are fought by teenagers controlled by giant robots.

    -- MarkusQ

  • Maybe it's just me, but this is gonna be the coolest game of MechWarrior EVER!!!!!
  • Recommended reading: Ender's Game [amazon.com] by Orson Scott Card
  • I've never understood why someone doesnt just make a battle-bot loaded with the quake engine, program some Quake AI into it, and put in a Map of the arena...
    Or at least, have a Quake-Style keyboard interface. Those RC controllers are bitchy
  • I would like my robots to be programmed with cheesy comic book style battle statements like "full re-route" or "utilize maximum force" or "sensors detect hostility" like microwave in the computer game "Freedom Force"
  • Obviously, this article is quite relevant, as we certainly have a need now to find our robots [slashdot.org] before they get away.

    And you thought trying to keep track of your hotkeyed units in Warcraft was hard...

    CC

  • Fifty years from now I expect all wars to be fought by giant robots controlled by teenagers.

    Oh yeah, that'll be just grand.

    *BOOM, giant robot bites the dust.*

    : |US|B0TL0RD `W00T!'
    : ]AoE[slaya `whore! u woz camping!'
    : |US|B0TL0RD `u axis lamerz just cant take it. go back to libya'
    : ]AoE[`fuk u! im getting kofi to ban u! U SUK BUSHS WANG!'

    So is how I see it...

  • I bet Ender was a whizz at Starcraft, taking on all those Buggers^WZerglings.
  • Fifty years from now I expect all wars to be fought by giant robots controlled by teenagers

    "The wars of the future will not be fought on earth, but in space, or at least on top of very high mountains, and they will be fought by robots. It will be your job to maintain and service those robots."
    Commandant, Rommelwood Military Academy

  • Hmm (Score:1)

    by a3d0a3m ( 306585 )
    Fifty years from now I expect all wars to be fought by giant robots controlled by teenagers.

    Did you ever see that unfortunate Robin Williams vehicle, Toys? They had the same idea, pretty much

    adam
  • Let's hope that they do not repeat the killer robot incident [onlineethics.org]
  • "Fifty years from now I expect all wars to be fought by giant robots controlled by teenagers." fifty years from now i expect to see all wars fought between robots and humans fighting for their freedom against robotic oppression.
  • Hmmm....am I the only one that has never read Enders Game and instead took this as a reference to an episode of SeaQuest?

Slashdot Top Deals

We are each entitled to our own opinion, but no one is entitled to his own facts. -- Patrick Moynihan

Close