Fold 'n' Drop Window Interaction 566
Mints writes "Following up on recent "Desktop Innovation" stories that have left some disappointed, I thought Pierre Dragicevic's exploration of Fold 'n' Drop warranted mention. Described as "a new interaction technique for seamlessly dragging and dropping between overlapping windows", Fold 'n' Drop allows the user to interact with layered or overlapping windows in a very intuitive manner. Refreshingly, Mr. Dragicevic provides both a sample implementation, in Java, and video demos. Mr. Dragicevic is a researcher in Human-Computer Interaction at Intuilab, Toulouse."
Keyboard Navigation Mouse Navigation (Score:3, Insightful)
Innovation or Eye Candy? (Score:4, Insightful)
Don't get me wrong, it looks really neat, but it's not terribly useful. Sounds like the kind of thing that would fit GREAT in Longhorn.
More trouble than it's worth? (Score:5, Insightful)
It's always nice to see new ways to interact, but I can't recall a single time this would have been useful in the past week. My memory can't recall much more than that, but the folding corners would certainly annoy me more often than it would actually be useful.
Cool Stuff ... for people with only one hand (Score:2, Insightful)
Re:It's already a solved problem. (Score:5, Insightful)
The Genie effect, translucent windows during a move operation, Expose, virtual desktops, dashboard, automator, tabbed browsing, and more are things for which similar results can be achieved by slightly clunkier or slightly less intuitive/clear/natural operations. They all offer significant improvement.
It strikes me that the window folding offered on the site represents exactly the same sort of thing. Yes you can achieve the same "effect" but you can do that on Windows via the taskbar. Neither expose nor the taskbar offer the very natural and intuitive method of flipping through the windows onscreen like flipping through a bunch of papers. The metaphor is much more clear. It is a significant improvement.
Apple is not the sole source of desktop innovation.
Jedidiah.
Re:Mac OS X Expose and Drag & Drop (Score:2, Insightful)
Re:It's already a solved problem. - Me Too! (Score:2, Insightful)
Unsurprisingly, Microsoft's way is the worst:
Re:It's already a solved problem. (Score:4, Insightful)
While I don't particularly like the grouping feature of the XP taskbar, if I have several windows open it's much more efficient for me to go straight to the corresponding button on the taskbar than to leaf through a stack of open windows until I found the right one. I prefer to use the ctrl+x/c/v, but I think even right clicking and selecting copy/cut and then navigating to the appropriate window is less cumbersome than holding down the mouse to shuffle through windows.
That said, there's more than one way to skin a cat. Some people may find the shuffling method to be preferential, and it would probably be beneficial to include such a technique in a new OS.
Re:It's already a solved problem. (Score:2, Insightful)
Are you honestly comparing a java demo to finalised software? Does the java demo have all the wonderful niceness of Quartz to do the graphics and compositing? No. Is the java demo optimised much at all? I expect not. Yes it is slow - it is supposed to give you an idea of how the concept could work, not a demonstration of how the guy expects it ot behave in a final product.
I would expect that when not using slow multiplatform raw java graphics libraries to do the graphics and compositing, but instead say Quartz or Cairo, or Avalon that it would be as quick as you can use. I look forward to this feature being available on Linux in the near future, and potentially in Leopard.
Use a bit of common sense and a little bit of imagination please.
Jedidiah.
Re:It's already a solved problem. (Score:3, Insightful)
You have to learn how to use a tool. A computer is a tool. Copy and paste are things you do when you use it. Nearly every program that supports copy and paste uses ctrl-c and ctrl-v and many keyboards even print "copy" and "paste" as hints. Ok, the Mac goes and uses the "Apple" key instead of ctrl, but it's the same idea.
Furthermore, they are very convenient buttons to press with your left hand while mousing with your right. Not perfect, utterly transparent design, but eminently pragmatic and *consistent*. That sounds usable to me.
Re:Keyboard Navigation Mouse Navigation (Score:3, Insightful)
Re:Innovation or Eye Candy? (Score:5, Insightful)
Re:Innovation or Eye Candy? (Score:1, Insightful)
Re:WTF???? (Score:4, Insightful)
Step three: Wait half a second for window "B" become the modal window and release.
What happens if the target window is completely obscured by the front window? If there is no overlapping edge for you to move towards and wait for focus to be given to the underlying windows?
From what I've seen, Mac OS really is the best with regard to user interaction tricks. It's the smoothest and best interface around. However, this new technique seems to have some advantages in terms of smoothness and it is intuitive. Clicking on a keyboard button may accomplish the same thing in the current Mac OS, but then again in Windows you can drag down to the taskbar and wait for that window to gain focus. It's just not as elegant as what's being proposed here. I, for one, think this sounds cool! You can push away the front window(s) and see what was previously obscured.
(Then again, I have not used Mac OS X that much and maybe what they already have is better than what is being proposed here... but still I think it's a neat idea worthy of consideration for any GUI.)
Re:It's already a solved problem. (Score:3, Insightful)
Firstly, I couldn't immediately figure out how to make the windows fold. Sure, when you move the mouse outside a window it begins to fold slightly. But it folds right back almost instanly, leaving you puzzled. I tried moving them mouse back when I saw the fold, but had no luck. It turned out that I just wasn't fast enough, but I didn't realize it until I went and read the site's directions, figuring I was doing it completely wrong. That definitely needs some work if my grandma is going to figure it out. I'd be surprised if she could even react that fast, not being trained on videogames since childhood. A longer delay is definitely necessary; however this would also be annoying in the common case that a window you didn't want to fold started folding.
The other major problem I had was that it's not easy to unfold large windows, or windows with other windows just behind them. It would be very hard to unfold maximized windows. And if you fold the window over completely it disappears and is impossible to get back. My grandma would be very disturbed if her windows disappeared (she's rather paranoid about losing stuff in the computer).
There's really nothing wrong with the "drag to the taskbar" approach. It's intuitive enough for people who are already familiar with the taskbar. For OS X's genie effect taskbar, it's so intuitive it hurts. The problem is that Windows implements taskbar dragging horribly, making you wait for no reason. The windows should flip instantly, as soon as your mouse is over the button. To avoid upsetting your window order, the windows should resume their original order afterwards (except the drag target should be on top so you can see, edit, or undo the results). And why the hell can't I drop things on taskbar buttons, Microsoft? You didn't change the cursor to indicate so. Surely it can't be too hard to provide an API call to register a drag handler for the taskbar button.
Actually, I dislike drag-and-drop as a UI concept in general, especially between different programs. Once you start dragging it's not always clear how to cancel; I generally move the mouse around randomly until I see the "no" cursor, but it's not always easy to find and sometimes I drop things unexpected places because I thought they wouldn't be "droppable". The results of drag-and-drop operations between windows are not predictable. Files might be copied, moved, or linked. Non-files are usually not supported, and when they are supported the results are often not exactly what you wanted. The position of things after you drop them is rarely what you intended because programs don't provide adequate feedback while dragging. Drag-and-drop is hard on the wrists if you have carpal tunnel.
Some of these problems could be fixed. I'd like it if you could release the mouse button after you start dragging; another click would release the hold, right clicking would cancel (or in some cases bring up a menu; canceling the menu would then cancel the drag). More importantly, though programs should provide *instant*, *completely accurate* feedback about the results of a drag operation. That means that the screen should look the same just before you drop an object you're holding and just after.
Wow, this turned into a major rant. Someday, I'll design my own OS, and fix all these dang problems.
Re:Not so useful for me. (Score:3, Insightful)
(Actually, the only application that I use regularily that doesn't support it under OS X is WMP.
Re:The stupidest GUI metaphor ever (Score:2, Insightful)
So I take it that you're always delighted when Clippy offers to format your document as a letter?
For that matter, are you delighted when your mother-in-law suggests that it would be quicker to make a left at the next intersection?
AI will be good enough to lay out my windows how I want them to work optimally about as soon as AI can actually do the work for me. Neural nets? You must be kidding, right? Predicting what a human is trying to do is hard enough for other humans, let alone computers.
That's not to say I don't agree that the metaphor is pretty ridiculous. Hey, it looks really cool. Everyone else in the office was looking over my shoulder. But would I actually want it in my window manager? Not likely.
Intuitive? (Score:2, Insightful)
Does look pretty cool though.
Re:Mac OS X Expose and Drag & Drop (Score:3, Insightful)
Target area is huge (you don't need to be precise with the folding) and the distance to move is minimised (just over the edge of the window and back). According to Fitt's law that's actually pretty damn good compared to over to the corner then all the way back to a small target.
Is folding going to be faster? In general it will mostly be about the same I expect, and in some cases it will be slower. It is also much more discoverable, and clear how to use it once you've started. Those are benefits that mitigate the speed loss. Will this replace Expose? No way. Would this be a damn good feature to add regardless of what expose might be able to do? Hell yes.
Jedidiah.
Re:Mac OS X Expose and Drag & Drop (Score:3, Insightful)
No thanks. I keep my left hand out of my pants while I'm using the computer, so I might as well use it to hit a key every so often.
Re:Innovation or Eye Candy? (Score:3, Insightful)
You're calling freaky window folding and flopping intuitive? I'm waiting for this supposed intuition to be applied to the desktop UI while my own mother can't even find the start button. Yeap. So intuitive.
If there is one thing I've learned, it's that ALL computer use is learned. People get "intuitive" mixed up with "ubiquitous" all the time. The fact that most everyone is familiar with one thing is supposed to mean that everyone else automatically gets it, as if knowledge was imparted through osmosis.
Honestly, what is more intuitive than learning to tell the computer exactly what you want by typing it in and having the comfort of knowing it'll do the same thing every time you type it (no tricks, gimmicks, or special cases to jump out at you)?
Re:Innovation or Eye Candy? (Score:3, Insightful)
Second, why do you have to steal the said free porn?
'Fold and Drop' is for losers... (Score:2, Insightful)
Uh... Excuse me... but have you ever told an end user that you can copy and paste a file? I have... and they're often dumbfounded.
End users, and by that I mean the 'those who need all the help they can get' loser end users, will eat this up. Why? Because the first time they try to drag a file from one window to another, they'll see the window fold... and either freak out, or understand what it's for. No instructions. With 30 seconds of experimentation, they'll understand. It's intuitive. Remember when 'intuitive' was the goal of every GUI? Yeah, me too...
So, we have the windows users: "Drag to the taskbar, bring up the window you need!" Our 'loser' user doesn't know this one.
How about the Mac User: "Expose does this already! And Springloaded Folders too!" Yeah, it does... Springloaded folders are in the right direction, and so is Expose. But this is better and eaiser than Expose, and on par to Springloaded Folders.
Then you have your traditional Unix user: "Focus Follows Mouse! DUH!" Or even better yet: "Mouse?! Whatever happened to useing the keybaord? If you have to use a GUI, use Ratpoison or Ion!" These guys give me the giggles... First off, I'll say yes... keyboards are much more efficient. Give anybody an on screen keyboard to type an essay, and they'll agree. But focus following the mouse is more anoying than useful to most all of our 'loser' enduers. As for the mouseless window managers... You forget that the target audience doesn't like the keyboard for anything save typing. If more than one button needs to be pushed at the same time, the user will tend not to understand. The only major exception would be Ctrl-Alt-Del, which has Windows users on Macs very frustrated from time to time.
Then you have these guys: "What a waste of my CPU and GPU! Can't we get rid of the eyecandy?!" Quit your whining. Today, our 'loser' end user can purchase a system from Dell with 2.5 thousand times the processing power of the computer that the Apollo astronauts used to get to the moon, And our user will be doing things much less important. As for the GPU, I don't belive the Java script is 3D accellerated, and it seemed to run just fine... IN JAVA mind you... This in C++ or C would run much faster still. If a few wasted cycles make the system easier to use and understand, I'd call those cycles well used indeed.
So quit your moaning. Give this guy the credit which is due: It's a very facinating idea, and I hope to see it implemented in all the OS's as an option. From watching the Video, then trying the Java... I'm hoping to get it an option on all my computers, be them Windows, Mac or Linux.
-Pathway
Re:Innovation or Eye Candy? (Score:2, Insightful)
A novice needs things to be consistent; this is not only prone to surprise triggering, above, but also, this is a very unusual effect for two-dimensional windows.
A novice needs things to be repeatable; until you understand that it is back-and-forth across the border of the window, it is a mystical mouse gesture. I feel this way about most mouse gestures. After years and years of playing Street Fighter, I've come to the belief that these are not things you want to entrust to serious work.
I think this is a neat idea, and I might like to use it myself, but I agree with another poster, who suggests that his novice users have enough problems with computer use. My mother cannot keep single- and double-click straight. What makes you think she'll get the hang of this?
Ethan
Incorrect (Score:2, Insightful)
Works in the sample Java applet [liihs.irit.fr], but not (yet) in OriModo [kmonos.net]. The Java applet has better "sensitivity" too, easier to fold one window, and to leaf through many.
Now someone just needs a way to apply this to all my Firefox tabs...
GUI Inconsistancies... (Score:5, Insightful)
I'm all for the improvement and innovation of interface design. Making a design intuitive increases productivity, even among power users who may do a particular action only once among a few thousand, and may not have it as ingrained as expected. That said, there are some major drawbacks to this design that may serve to make it less productive and even less intuitive.
Counter-Intuitive Metaphors
Metaphorical abstractions for computer objects only work if they have a clear representation of being similar to object they represent. While some windows (text boxes, for example) have a clear similarity to being a leaf of paper, many others do not; directory windows seem fairly unrelated to 2d objects: they contain multiple objects inside of them, likening them more to being a box or drawer, some 3d abstraction. Thus it is not only counter-intuitive to "fold-over" an object which has depth, but also brutally forcing a metaphor onto an object of which could suggest a completely alien mental abstraction from the one a user original envisioned. For this reason, almost all interface references to real-world objects are either extremely obvious or very broad in definition. The "focus" metaphor works, for instance, because you can bring any object (one with depth or no depth) and put it on top of another object, thus bringing it into "focus" or plain-sight; it is an extremely simple and all-encompassing concept.
Temporality and Spacial Complexity
The second problem with this method is its inherent temporality. Most GUI operation requires no timing, and in the rare cases that timing is required (ie: double-clicking, hovering over spring loaded folder), the operation is extremely simple and requires no precision. The one exception is double-clicking, and you can witness its result by watching any surface user fail to open a folder because they can't keep the mouse still while clicking the left mouse button. The folding operation illustrated here, on the other hand, is an extremely complex operation that takes some very precise timing. Even I, an experienced computer user (as we all are), had to practice it many times to double-back on my mouse movement fast enough to correctly "fold-over" a window. Since windows move and change in organization, the operation is slightly different each time it is performed. I can already tell that even if it the operation becomes somewhat natural, I'll always continue to miss on occasion because of it's complexity. And if I'm having trouble with it, I can't imagine what it would be like for my parents!
UPDATE: I had my mother test it out to see if a surface user could cope with it, and after struggling with it for a few minutes, finaly gave up.
Accidents and Set-backs
The third problem I for-see is that folding can easily occur unintentionally and is difficult to undo. Spring loaded folders and "snap-to" focusing work well because their actions inherently require a very specific action: going over a folder and waiting for about a half a second for the window to pop up. Since the cursor is going to be generally moving while dragging objects, a half-second wait over a folder or partly obscured window is abnormal and requires intentionality. Even then, it is as easily (if not more easily) reversed as it done by simply moving off the newly focused window. With folding, on the other hand, it's easy to see how any quick movement during a drag could activate the effect, and when the process of folding is started, it takes an even more complex spacial action to set it back, that being the looping around and back onto the fold from the other side.
Just a few thoughts on intuitive interface design, using this as an example of what works and what doesn't.
Re:UI innovation and the Slashdot audience (Score:5, Insightful)
I wrote a few posts a few days ago on this, but I'll repeat...
As much as people talk about innovation, few really want such a thing. That's how it has always been.
Instead of innovation, people want familiarity. Which is why many people years ago did not want to move from text console to X11 when hardware and drivers were finally reasonable. I was one of those people, sadly. You couldn't get me to touch an xterm (or rxvt, my preferred). That is, until I discovered those nice terminal fonts and how it was possible to change the default xterm colors to that of a VGA textmode terminal. That is what I still use today.
I try to be as open-minded as possible, but I catch myself doing those same things today. I've had many discussions with people who claim to want innovation when they really want upgrades to the things they already use. There isn't much innovative about switching from devfs to udev, etc. yet quite a few act as if innovation occurs often.
Given the choice between backwards compatibility or innovation, hardware and software manufacturers will always choose backwards compatibility. Only because that is what the end-user always wants.
Re:I dunno (Score:4, Insightful)
Re:Innovation or Eye Candy? (Score:2, Insightful)
When was the last time you moved or copied text from one paper to another with moving your hand over it (like you would move mouse), pressing your index finger down, "folding" your papers by moving your hand and lifting your index finger up? I'm not trying to be mean here, but sometimes you can't just mix real world physical objects and computer world virtual objects and especially their functionalities.
But this folding gives me an idea of another kind of drag'n'dropping, which I think I'm going to implement some day. But I'm not telling what it is until I have patented it
Re:Interesting (Score:1, Insightful)
This could partly be due to the Java demo, but even a bad demo can't change the fact that the idea is totally non-intuitive.
The demo itself was terrible. Once I'd actually worked out what you had to do[1], the demo behaved eratically. I could "Fold" the top two Windows but others wouldn't seem to fold at all. Apparently you can "Fold" a Window out of existence (Never done that to a peice of paper!) The action of unfolding a bunch of Windows when the action was cancelled was jarring and distracting (I lost track of which Window I was looking at every single time)
This is a poorly thought out non-solution to a problem that offers nothing but a pretty transition effect and a small nod to the mouse-gesture crazies.
[1]: Oh, I have to be dragging something to activate the Folding? Why? Seriously, think about it a little. Why shouldn't this work just for folding Windows out of the way "normally"?
Re:Keyboard Navigation Mouse Navigation (Score:3, Insightful)
Re:Keyboard Navigation Mouse Navigation (Score:3, Insightful)
Took me about 5 minutes between sitting in front of Windows (95, mind you) for the first time and getting this message. If it happens so often that you feel the need to design a message box around this user behaviour, well gee maybe users really do want to be able to drop items onto taskbar buttons and you should just let them? Morons.
(Yes, I'm aware that you said hovering until it becomes the active window. It's still a stupid interface design.)
No! Wait! This rocks! (Score:3, Insightful)
confusing means with end (Score:3, Insightful)
Really new users find drag an drop about as intuitive (or less) as mv dir1/file1.txt dir2
Which to use depends on the person using the computer. Options are good.
Personally, I'll take keyboard use, its an order of magnitude faster, *for me*. I can do a keyboard shortcut in less time than it takes to remove my hands from thekeyboard to grab the mouse, let alone move and manipulate the mouse. If I'm just lounging out, keyboard out of reach, surfing the web, then I'll use the mouse because its more convienent.
Single and double click (Score:3, Insightful)
It's hard to explain. "Double-click invokes programs." Invokes? What the hell does that mean to the average person?
Paradoxically, single-click on a link causes a new web page to be displayed. That sure looks like something got "invoked", doesn't it?
How about all those apps that do something unexpected when you select an entry out of a combo box? Isn't the metaphor supposed to be SELECT, followed by PUSH A BUTTON?
So it's not all that easy to explain when you're supposed to single-click and when you're supposed to double-click. And, the fact that Windows misses some of the double-clicks you do because of the stupid-ass way double-click is implemented at the message level doesn't help, either.
Oh, and let's not forget the fact that Windows drops internal messages occasionally anyway (about 1 in 100,000 the last time I measured it, in case anyone cares), so remember that the next time you decide to create your own message class and "trust" Windows to deliver your messages.
Re:Keyboard Navigation Mouse Navigation (Score:5, Insightful)
Linux: the mouse is only good for click, drag, and select/copy. Users believe the mouse is a useless add-on. On Linux, I agree.
Windows: good for getting those right-click menus. Also the only way to do things that don't have obvious keyboard shortcuts - preference dialogs, toolbar buttons, etc.
Mac: Drag and drop everywhere. Bind the middle button to Expose. Eventually you just keep your hands in the Quake position: left hand on the kb, right on the mouse. You know, a GUI.