Video Chat Via Transparent Desktop Overlay 204
Jason0x21 writes "Wired News has an article about UNC Comp. Sci. researchers developing a transparent desktop overlay for video conferencing, allowing remote coworkers to literally point and interact with things on your screen. The researchers say that Apple's Quartz graphics engine let them go from idea to prototype in 'about 45 minutes'. Windows versions predicted in the future."
Similar piece of tech... (Score:5, Interesting)
It's a great presentation tool to liven up a powerpoint and avoid the need to have to walk accross the room to get the next frame. Furthermore, playing solitare with foot-high cards is quite fun.
Re:Similar piece of tech... (Score:3, Interesting)
However, they're great for graphics work. This fact is offset by the fact that they suck royally in Quake.
Re:Similar piece of tech... (Score:4, Funny)
Re:Similar piece of tech... (Score:3, Interesting)
The touch sensitive ones kind of suck since they aren
Re:Similar piece of tech... (Score:5, Funny)
Re:Similar piece of tech... (Score:3, Funny)
(You mean there are other Guidos?)
Re:Similar piece of tech... (Score:2)
You gave a far better explaination than the grey-haired head teacher...
Mac ahead of Windows on graphics, again... (Score:5, Informative)
Particularlly, you can't do any blending against windows that are being drawn with DirectX/DirectDraw which is the way that any program that wants to approach full-motion video or 3D graphics has to do things. And that's what prevents Windows from handling this application.
Mac's OSX is a lot cleaner in this department because in their universe there are no exceptions to the rules... everything passes through Quartz, so there's a chance to capture and play with anything on the screen. DirectX and DirectDraw are painted onto the screen after all mortal windows are drawn in Windows, and that's why there's no chance to add an overlay to them.
Re:Mac ahead of Windows on graphics, again... (Score:5, Interesting)
This is especially noticeable if you make a window transparent, then open the Task Manager (always on top) and drag the transparent window around. Very ugly..
And I haven't seen any way to make specific controls on Windows transparent alone or the window transparent alone (and leaving the controls opaque.
Re:Mac ahead of Windows on graphics, again... (Score:3, Interesting)
Re:Mac ahead of Windows on graphics, again... (Score:5, Informative)
Re:Mac ahead of Windows on graphics, again... (Score:2, Informative)
Just don't expect 2D and 3D pipeline windows to intermix with translucency. While this system would be *possible* under Windows, we feel, at this point in time it is not *practical*. Three months were spent attempting a proof of concept under Windows. The prototype on the Mac took 45 minutes.
Longhorn's new graphics system will bring it to parity with Quartz, and it should then be equally feasible there as well.
not really (Score:3, Interesting)
Transparency in window systems is an idea that goes back almost as long as window systems have been around. People were even asking for it in the earliest versions of X11.
The only reason it hasn't been implemented more widely is because hardware hasn't really been up to it and applications didn't need it. Those applications that really did need it just used special graphics and visualizat
Re:Mac ahead of Windows on graphics, again... (Score:2)
Mac's OSX is a lot cleaner in this department because in their universe there are no exceptions to the rules... everything passes through Quartz, so there's a chance to capture and play with anything on the screen. DirectX and DirectDraw
Re:Mac ahead of Windows on graphics, again... (Score:2)
45 minutes? Isn't that more cocoa? (Score:2, Insightful)
Re:45 minutes? Isn't that more cocoa? (Score:5, Informative)
But yes, Cocoa made it much easier to do so.
Story text in case of /.ing (Score:5, Informative)
For a couple of decades, researchers have tried to blend shared workspaces -- systems that allow two or more people to work on the same document -- with Internet video-conferencing systems, with little success.
Now researchers at the University of North Carolina at Chapel Hill have designed a new system that cleverly blends a video-conference feed with a transparent image of a computer desktop into one full-screen window.
Called Facetop, the system simultaneously transmits a video feed of users along with a shared, transparent image of the desktop. It allows two colleagues to work on the same document, Web page or graphic, while communicating face to face.
The system also tracks the position of the users' fingertips, which can control a cursor. As well as operating the shared desktop -- opening and closing files or selecting text, for instance -- the collaborators can use natural pointing gestures to communicate ideas about the document.
Developed by David Stotts, an associate professor of computer science, and graduate student Jason Smith, Facetop was conceived for collaborative tasks like programming or editing text. But the researchers say it has obvious uses in other areas such as medical imaging or remote teaching.
"So far, from the feedback we've received, it works fantastically," said Smith. "It's a very natural interaction. You can see the facial expressions and all the nuances of face-to-face communication."
"It's spectacular technology," said Robert Gotwals, associate director of Chapel Hill's Morehead Planetarium and Science Center, who saw a demonstration of an early version. "I've done lots of video-conferencing work. This is pretty cutting edge. It's a fast-moving field and the stuff David (Stotts) is doing is pretty cool."
The system can also be used for delivering lectures or PowerPoint presentations: The speaker is projected in the background of the document allowing her to point out bullet points or important passages. According to Smith, users easily switch attention between the subject and the desktop.
"The brain is really good at picking out what part of the screen the person is interested in," said Smith. "It's like being in a room full of conversations but having no trouble paying attention to only one.... People adapt to the system really naturally."
Facetop may also be used to as an alternative to the mouse, for controlling a machine simply by pointing with a finger.
The system is implemented in Mac OS X and is made possible largely by the system's Quartz rendering engine, which can make any part of the interface transparent. Thanks to Quartz, a quick prototype was whipped up in about 45 minutes, Smith said.
A PC version will likely be delayed until the release of Longhorn, the next major version of Windows, due in 2006, which will include a similar graphics subsystem.
The system is fairly inexpensive; it has been implemented on a pair of Apple PowerBooks and two $100 FireWire cameras. So far it has been tested only on Ethernet networks and not the Internet, though the researchers say there's no reason it shouldn't work just fine. They are also trying to hook it to Apple's iChat instant-message/video-conferencing software and other similar systems.
Facetop was initially developed for "pair programming," an increasingly popular form of collaborative coding that pairs programmers in teams of two: one to program, the other to suggest and correct. Stotts said programmers normally sit next to each other, and he has been interested for some time to see whether they could collaborate over the Internet.
According to Stotts, pair programming -- sometimes called extreme programming -- is fast and effective and is becoming increasingly popular for small projects.
The idea for Facetop occurred to Stotts and Smith accidentally. Instead of a computer monitor, Stotts projects his
Re:Story text in case of /.ing (Score:2)
Damn Karma whores.
Where are the screenshots? (Score:2, Interesting)
Why boast how easy it was to get it happening then not showing it happening?
odd
Re:Where are the screenshots? (Score:1, Funny)
Re:Where are the screenshots? (Score:1)
They are chilling on the LHS of the screen.
Re:Where are the screenshots? (Score:5, Informative)
http://a1112.g.akamai.net/7/1112/492/2002091464/w
Re:Where are the screenshots? (Score:4, Insightful)
Re:Where are the screenshots? (Score:2)
I'm wondering is that isn't doctored. He's looking straight at the camera... unless they've somehow come up with one-way displays with cameras behind them as well... and that would of course be the bigger story.
Re:Where are the screenshots? (Score:2)
Re:Where are the screenshots? (Score:5, Informative)
That's rather the *point*.
Trust me, it would have been much easier to take a picture of a reflection on the screen surface than develop the bloody thing.
Prior research (Score:4, Interesting)
But that seems to miss the point (Score:2)
Sounds like what you were working on was just a shared workspace?
Re:Prior research (Score:2)
Yes, it's sad that it took three years.
Software developed by, or on contract from, the federal government belongs to the public domain in the US, if I remember correctly. If that DoE software package had had its source opened up and made available in a timely fashion, a civilian company would not have had to redevelop their own system from scratch.
Not exactly (Score:2, Informative)
What i'd like to see is a voice controled program, instead of hurridly bending down to click the mouse at a conference, you simply say 'back', 'foward', 'pause', or even program in new words through a macro system built into the program. Oh and don't try and steal it, thats my damn intelectual property now, hah!
Re:Not exactly (Score:2, Interesting)
I once.. (Score:3, Funny)
Re:Not exactly (Score:2)
They had a complete voice recognition system which was IIRC sold seperately though (but integrated nicely into the OS). One of the two main selling points was presentations.
The other was word processing which was the make-or-break point and where the big problem lied. It just too long to train the application and, more importantly, to actually compose a quick letter using voice recognition. I remember using it at presentations though and with
Re:Not exactly (Score:2)
Get some decent computer vision going, and since your hand can be overlayed on the screen, just use your hand as a mouse. much like tom cruise's setup in minority report.
Linux version with freedesktop? (Score:1, Interesting)
actual project link (Score:5, Informative)
FaceTop [unc.edu]
what about your background? (Score:2, Interesting)
Re:what about your background? (Score:5, Informative)
First off, the translucency is adjustable. Looks too cluttered? Make it more faint. Secondly, it's much like being in a room full of conversations at a party - you select particular conversations to pay attention to, and the rest just 'fade away'. In this case, when the user turns their attention to the document content, they don't notice the video, and when they concentrate on the video (either for hand motions or interaction with a remote user), the document content is ignored. The brain is much better at this sort of thing than most people realize.
...sgub emos llits s'erehT (Score:5, Funny)
!siht ekil skool gnihtyreve weiv fo tniop rieht morf ,yletanutrofnU
Re:...sgub emos llits s'erehT (Score:4, Informative)
Re:...sgub emos llits s'erehT (Score:4, Informative)
I just put up a stupidly simple FAQ of sorts at http://www.cs.unc.edu/~smithja/facetop/index.html and will be updating it this morning.
Please Mod Parent Up! (Score:2)
Talk about an uber-trojan (Score:4, Insightful)
Re:Talk about an uber-trojan (Score:2)
Re:Talk about an uber-trojan (Score:3, Funny)
1. Your boss's face stares back at you your entire time at work and you can't determine whether the image is live or pre-recorded.
2. You browse a porn site and instead of popup ads, you get a Real Live Operator offering a different sort of popups.
3. You get 0Wn3d but don't feel so threatened after seeing the pimpled face of 14-year old.
4. Instead of visiting a religeous site and using your web browser to submit prayers in a t
Re:Talk about an uber-trojan (Score:2)
any trojan would be able to do it(move icons, close windows - in fact some spyware/adware/viruses do that already) without this in the os.
and did you even look at the screenshots/article? obviously not, or did you intend that there would be some guy personally moving the icons on normal victims screens? that would be quite expensive for advertising purposes.
Windows version when? (Score:4, Interesting)
Re:Windows version when? (Score:2, Informative)
Like you said, this isn't a Windows Mac pissing contest. Its a cool new
Re:Windows version when? (Score:2)
However, that could probably still be done, using a hardware DirectDraw/Direct3D video overlay to draw the webcam feed over the desktop with reduced opacity or per-pixel alpha.
VNC? (Score:2)
But it's not like this technology at all. Sorry.
Heaven Forbid! (Score:5, Funny)
Re:Heaven Forbid! (Score:2, Insightful)
Re:Heaven Forbid! (Score:3, Informative)
For example, the current project I'm working on involves 3 companies spanning 5 or 6 states in the US. We can't just meet face-to-face without spending a hell of a lot of money.
And next week I'll be working on something else for a completely different company.
This is how stuff gets done. It allows talent from all over the globe to be used for minimal cost.
Re:Heaven Forbid! (Score:2)
Only 10 years behind (Score:5, Informative)
I saw the concept videos [mit.edu] in my HCI class at the time. They went through all the various issues of pointing alignment, video flipping and the like.
Re:Only 10 years behind (Score:4, Informative)
1) ClearBoard's conceptual model was two people standing on either side of a pane of glass. Ours is a much simpler view... two people sitting side by side. We have no issues requiring us to flip document content for instance. It is a small but important difference in how it drives the implementation.
2) ClearBoard required expensive and cumbersome hardware. FaceTop requires a $100 FireWire camera. Well, and a Mac.
3) ClearBoard was designed to be integrated with specific applications. FaceTop becomes an input device, much like a mouse replacement, and thereby can work with any application on your system. We generalized it out, and it became much more powerful.
Re:Only 10 years behind (Score:2)
patentable innovation of FaceTop could be,
given ClearBoard's prior art.
Re:Only 10 years behind (Score:2)
The FAQ [unc.edu] also illustrates that the number of people interacting is the number of people shown - you also see yourself, something ClearBoard did not do. The FAQ explains
Re:Only 10 years behind (Score:2)
Hopefully, if you are right, the PTO will deny the patent. (I'm not a huge software patent fan myself, I just disagree with you on the uniqueness test with relation to ClearBoard.)
Re:Only 10 years behind (Score:2)
I have used a ClearBoard and it was cool and I wanted one immediately. I suspect your app will be another good reason for me to get a Mac again.
I also was a little involved with Muse 2000, cscw visualization and collaborative simulation/cad construction software in an n-d
Re:Only 10 years behind (Score:2)
As for what is patentable, I am unable to comment on that until the patent process is complete. (Yeah, it's not my first choice either, but I'm bound by the legalities of my situation.)
You're right, there is a lot of good work out there that we've uncovered since starting this project... to be honest, I'm not a CHI guy, I'm a software engin
Re:Only 10 years behind (Score:2)
Re:Only 10 years behind (Score:2)
> on either side of a pane of glass. Ours is a much simpler
> view... two people sitting side by side.
No, see ClearBoard-2 [mit.edu] (scroll down):
"To overcome these problems in ClearBoard-1, we decided to design a new computer-based prototype, "ClearBoard-2"."
> 2) ClearBoard required expensive and cumbersome hardware...
Adding newer hardware to an existing design is hardly innovative. Nothing pertinent to a patent there.
> 3) ClearBoard was de
Re:Only 10 years behind (Score:2)
> 1) ClearBoard's conceptual model was two people standing
> on either side of a pane of glass. Ours is a much simpler
> view... two people sitting side by side.
No, see ClearBoard-2 [mit.edu] (scroll down):
"To overcome these problems in ClearBoard-1, we decided to design a new computer-based prototype, "ClearBoard-2"."
Sorry - I didn't notice you wrote _conceptual_ model. I thought you were referring to the ClearBoard 'Version 0' on that website.
Re:Only 10 years behind (Score:2)
Seems like Ishii's work should invalidate the University of North Carolina patents.
From the article:
Re:Only 10 years behind (Score:2)
Screenshots (Score:5, Informative)
Screenshots [weblogs.com]
Peak Performance (Score:4, Informative)
-Ian
Re:Peak Performance (Score:2)
Go figure (Score:4, Interesting)
Re:Go figure (Score:3, Insightful)
Re:Go figure (Score:2)
Re:Go figure (Score:2)
It would ottally depersonalize the support experience. No longer would customers deal with people, some faceless entity would just grab their computer and fix it.
Not to mention, and I think this is important--if you can fix a user's computer without leaving your desk, that same job can be done for less in India. Remote Is Not Your Friend--much better to retain the comfort of face-to-face interaction, and to build interpersonal relationships; call it a value add.
If you train your users to be comfortab
Re:Go figure (Score:2)
Adding 3D to it? (Score:5, Interesting)
I don't really know if it would be useful, but perhaps it is cool to lean a window so you can see your partner while keeping an eye on the app content.
Anyway, beeing so far from the world as *I* am (yep, there are places on the south of the globe), where the bandwidth is kinda expensive, i can tell that i'll not be using this kind of technology for a while...
--krahd
Hmm... (Score:5, Funny)
Wait! How can this work? (Score:3, Interesting)
the image of your hand (despite the touch screen) on your collaborator's screen has to be computer generated. Or am I missing something here?
If the hand is CG, then all we have is a glorified cursor (but this too would be a pretty good hack if they got it done in 45 mins).
But wait, those pics don't seem to show the hand pointing in the right direction either!
Re:its a reflection in the glass (Score:2, Informative)
Yes, that's precisely why it works - it's like you're looking into a mirror. Raise your hand to move the cursor up. Move it left, cursor goes left. It's just that simple.
Comment removed (Score:4, Insightful)
Re:How does this work with the camera facing you? (Score:2)
Doesn't this mean that what you see is actually a mirror image of yourself, and that in order to guide the cursor to an icon, you have to manipulate your on-screen finger to the right place and then flick it? (As opposed to a touch screen, where your REAL, not virtual, finger actually does the clicking).
You do realise that the camera is digital and merely passes a stream of numbers into the computer? It is relatively trivial to reverse the order of the numbers in software. "Wal
Re:How does this work with the camera facing you? (Score:2)
Re:How does this work with the camera facing you? (Score:2)
The reason I put it in quotes is because I deliberately spelt it wrongly in a crude attempt at anti-slashbot irony. I'm not quite that stupid. I note that you post as AC.
Re:How does this work with the camera facing you? (Score:2)
Re:How does this work with the camera facing you? (Score:4, Informative)
As for ease of use, it literally takes people about two seconds to calibrate their hand motions to the cursor movement, and they're off and running. It's exactly like you're standing in front of a mirror (assuming the camera is in front of you), and gesturing... the visual feedback you get from your own image is the key. The transparency lets you see both your 'reflection' and the document content simultaneously.
Don't worry, we're seeing a lot of people confusing the single-user mode (one head on screen) with the video-conferencing mode (two heads on screen), simply because they're not used to video conferencing including themselves.
Try out the interface at home WITHOUT A COMPUTER! (Score:2, Interesting)
If you don't have stickers, make a few small circles on the mirror using your girlfriends lipstick.
Now step about 3 feet away from the mirror.
Move your finger so that when you look in the mirror, it looks like you are touching the stickers but you don't physically do so, it just looks like you do to your eye.
Notice that you can do this regardless of your angle to the mirror, you just have to adjust your finger.
Now imagine that the stickers are icons on
Re:Try out the interface at home WITHOUT A COMPUTE (Score:2)
Bump that puppy up, mods, if you don't mind.
Hey all, one of the co-inventors here with a FAQ. (Score:5, Informative)
I'll be adding material to it through the morning as issues pop up, but these are the ones we've seen the most of this weekend.
Re:Hey all, one of the co-inventors here with a FA (Score:3, Informative)
It would be nice to just disseminate the thing, but I don't legally own it to do so.
Re:Hey all, one of the co-inventors here with a FA (Score:2)
We are in the endstages of the patent *process* (applications formally filed, etc), and until I get the all-clear from the suits, I am not going to say a blasted thing outside of an NDA that hasn't already been cleared through channels such as academic publications and interviews. I like my grad student posterior unkicked by wingtips.
This has been done quite well already... (Score:3, Informative)
I've tried it and it's pretty cool. It's great so see the expression on your opponents face when you roll your army of tanks into his left flank when he's least expecting it.
Re:This has been done quite well already... (Score:2)
Combine this with 3D LCD technology.... (Score:2)
Shame It's Patented (Score:2)
I am not an Open Source/Free Software zealot, but it does irritate me somewhat when publicly funded institutions patent research projects =)
I also mean no disrespect to the develop
Re:Shame It's Patented (Score:2)
eye toy (Score:2, Interesting)
This is a better use of the idea in my opinion. I'd like to use this to replace a mouse, plus the collaberation use looks great. Kudos to the ones who put it together!
Can we do this with X11?
The real AHA! here (Score:2, Insightful)
Re:The real AHA! here (Score:2)
Check the FAQ here [unc.edu] for a rundown of what's actually going on.
You see the other person *and yourself*. It's as if you're sitting side by side, working at the same keyboard. If either of you lifts your tracked fingertip into the camera view, the cursor is controlled by it. Either user can control the cursor, and edit the shared document(s).
And you're right, there is very little confusion as to what's going on - most people take to it immediately.
Re:The real AHA! here (Score:2)
Re:I don't get it... (Score:2, Informative)
Without video conferencing, you lose a lot of the nuances of human communication.
Stotts' work in distributed pair programming almost invariable resulted in "We really wish we could *see* them..." which led to video conferencing, which led to "We still have to verbally describe things we could just point to."
Also, VNC clones the same desktop to both machines - which makes traditional video conferencing... difficult.
With our approach we can clone the entire scree