Forgot your password?
typodupeerror
GUI Software Technology

Shuttleworth Suggests 1-Way Valve For User Experience Testing 757

Posted by timothy
from the why-isn't-this-intuitive? dept.
darthcamaro writes "No surprise but Ubuntu's Mark Shuttleworth has come out swinging in favor of the Linux desktop. Speaking at Linuxcon yesterday he detailed the things that he thinks Linux requires in order to win the desktop wars. Those include: co-ordinated software releases, better quality and design, some user experience testing and oh yeah, a dose of 'shut the f*** up' too. During his keynote, he extended an invitation to any open source application to submit their software for testing by user-experience experts. The sessions would be recorded for posterity, and the developer would not be able to interact with the user. "'If the developer is in the room, they have to say nothing. It's the shut the f*** up protocol,' Shuttleworth said. 'You sit and watch someone struggle with the software that you've so lovingly produced.'"
This discussion has been archived. No new comments can be posted.

Shuttleworth Suggests 1-Way Valve For User Experience Testing

Comments Filter:
  • by Anonymous Coward on Thursday September 24, 2009 @04:19PM (#29532793)

    Users always ruin the best software.

    • by Romancer (19668) <romancer@ d e a t h s d o o r .com> on Thursday September 24, 2009 @04:26PM (#29532901) Journal

      What did he just tell you? STFU!

  • Kudos to him! (Score:5, Insightful)

    by jackharrer (972403) on Thursday September 24, 2009 @04:22PM (#29532839)

    He knows what he's talking about. We don't need more RMS but more people like Shuttleworth. Pragmatically minded, not focused only on ideals. If somebody wants follow only ideas I suggest Green Peace or monastery.

    • Re:Kudos to him! (Score:4, Insightful)

      by Jah-Wren Ryel (80510) on Thursday September 24, 2009 @04:25PM (#29532879)

      We don't need more RMS but more people like Shuttleworth. Pragmatically minded, not focused only on ideals.

      Right. There are definitely not enough people to go around.
      Damn those idealists, sucking up all the available people, keeping them from getting anything done.

    • Re:Kudos to him! (Score:5, Insightful)

      by Anonymous Coward on Thursday September 24, 2009 @04:30PM (#29532943)

      Yeah! Ideals are for losers!

      FFS.

      Did it ever occur to you that without RMS there would be no Shuttleworth?

    • Re:Kudos to him! (Score:5, Interesting)

      by Darkness404 (1287218) on Thursday September 24, 2009 @04:34PM (#29533013)
      We -need- RMS though. Without RMS we just have a bunch of people wanting to get stuff for free. Heck, without RMS and the GPL, Linux would not exist, Linux as in the kernel itself. Chances are it would have been licensed under an obscure license and died due to a non-commercial or other clause. It was only due to the GPL that the kernel was released under a typical license.
    • Re:Kudos to him! (Score:5, Insightful)

      by node 3 (115640) on Thursday September 24, 2009 @04:38PM (#29533075)

      This is Linux, we can have, and need to have, both.

      There will be free Linuxes, like Debian. There will be "pragmatic" Linuxes, like Ubuntu. There will be all sorts of Linuxes in between.

      Linux requires both the RMS types and the Shuttleworth types in order to both survive (RMS) and grow out of its niche (Shuttleworth).

    • Re:Kudos to him! (Score:5, Insightful)

      by phantomfive (622387) on Thursday September 24, 2009 @04:40PM (#29533101) Journal
      People who forget the lessons of the past are bound to repeat them. RMS is right, and it is only upon the foundation that he started that Mark Shuttleworth has anything to stand.

      Not saying Mark Shuttleworth is doing a bad job, but when you start saying things like, "It's either Richard or Mark! One or the other!" you've kind of gotten off base. They both have good things to say and actually for the most part are in agreement.
  • by B5_geek (638928) on Thursday September 24, 2009 @04:23PM (#29532851)

    Ok, sounds like a cool idea. I would LOVE the Amarok2.x devs to sit in on that session.

    • by dstar (34869) on Thursday September 24, 2009 @04:33PM (#29533009)

      I would love for the Gnome developers to sit in on that session.

      And then be beaten with sledgehammers until they understand that the goal should not be 'unconfigurable' but 'no configuration needed 90% of the time, and configurable the remaining 10% of the time'.

      • by Fallingcow (213461) on Thursday September 24, 2009 @04:51PM (#29533251) Homepage

        I'm a coder, gamer, and all-around power-user. I've been using Linux for years, including 2-3 years of using Gentoo exclusively, back before it had any sort of gui installer.

        In all that time, I've only had Gnome not let me do something (or make it overly difficult) twice: once was when they went to "spacial" (I think they called it) handling of folder-opening in Nautilus, which was only a slight pain to fix and which, AFAIK, has been switched back to a not-retarded default anyway, and setting each virtual desktop to a different background, which I'd still like to be able to do but which really isn't that big a damn deal.

        What exactly do all these "Gnome won't let you configure anything! KDE 4Evar!" people want to be able to do with Gnome that they can't?

        • by dstar (34869) on Thursday September 24, 2009 @05:08PM (#29533425)

          I wanted to tell NetworkManager to do something specific (IIRC, use a specific DNS server rather than the one handed out by the DHCP server on my DSL gateway, but it's been a year or so) and couldn't. When I opened a ticket about it, it was closed WONTFIX with the notation that the idea behind it was zero-configuration and adding the ability to configure it to do this was therefore unacceptable.

          I want gnome-terminal not to eat my right-clicks. People have been asking for that for *years* and are constantly told that the Gnome developers know better than they do about what they need.

          • Re: (Score:3, Informative)

            by Fallingcow (213461)

            I wanted to tell NetworkManager to do something specific (IIRC, use a specific DNS server rather than the one handed out by the DHCP server on my DSL gateway, but it's been a year or so) and couldn't. When I opened a ticket about it, it was closed WONTFIX with the notation that the idea behind it was zero-configuration and adding the ability to configure it to do this was therefore unacceptable.

            Ok, that's true. NM's better than the awful GUI network config programs that came before it, and the alternative

    • Amen to that. Fortunately, there is a godsend for Ubuntu users: Amarok 1.4 series PPA [launchpad.net]. You just add it to your package sources and install "amarok14". Thank you Bogdan Butnaru.

  • by Nick Number (447026) on Thursday September 24, 2009 @04:24PM (#29532863) Homepage Journal

    and oh yeah, a dose of 'shut the f*uck up' too.

    Wow, it's a good thing that asterisk was there. Somebody might have seen something profane.

  • To be so lucky... (Score:5, Interesting)

    by Monkeedude1212 (1560403) on Thursday September 24, 2009 @04:27PM (#29532907) Journal

    I've done a bit of software dev here and there, and I've never had the luxury of being near the users when they first prop it open.

    For that reason, I've developed a habit of showing a beta to a nearby co-worker, or a friend, and ask them "Check this out."

    And when they say "What is it?" - I haven't done my job right.

    • by Kell Bengal (711123) on Thursday September 24, 2009 @04:33PM (#29533005)
      This is a good point - if software doesn't explain itself, then it is broken. I believe this holds all the way from the top level to the basics. If the architecture of the system isn't well signposted and comprehensible, it fails. If an icon meaning is murky and there are no tooltips, it fails. Now you always have to assume some basic level competence on the part of the user (eg. knowing to type man to get program info, or knowing how to click with a mouse) but once you're part that, there is no reason why programs can't be self-explanatory, or at the very least self-documenting. I don't know how many times I've torn my hair out because the 'Help' menu's only item was "About".
      • Re:To be so lucky... (Score:4, Interesting)

        by Monkeedude1212 (1560403) on Thursday September 24, 2009 @05:12PM (#29533471) Journal

        Well exactly. Someone up there in the comments mentioned how Open Source Developers like to "Scratch their own itch" - which in my opinion is really the wrong way to tackle a problem. What alot of developers don't realize is that the actual design process (which should be done first) can be done with very little input from the developer. Well, the blame isn't squarely on them, the users also need to be clear and concise about what they want and they need to be able to present it to a developer - in the same way the developer has to present their product:

        A Developer can't be expected to know everything about how the user does their job - but they're expected to make a program to assist in that process. Likewise a user can't be expected to know everything about how a program works - but they're expected to know how it works when its done.

        So there is this huge middle ground where either:
        A) The user is so confused by what does what, because the Developer took it upon himself to make an amazing program that does it all
        or
        B) The user is not happy with what the functions are doing - or its missing functionality, or some logic is missing - which ends up being blamed on the Developers for not making it properly and they have to go through it all again.

        It usually all boils down to people not telling the Developers enough, or the Developers are assuming that they know what to do and don't ask questions. The solution is just better communcation. If you can, get a Visio Diagram going, maybe some flow charts, anything and everything to help you lay down the design of the project. Designing is really a 2 way street, it needs to be done by the user just as much as it does have to be done by the developer, maybe even a little more so on the user. Once Design is down - Development becomes an EASY process that can be done in DAYS instead of weeks. It becomes like High school physics where you have the formula sheet and you just plug in the numbers to get the answer. (Assuming you were good at high school physics. I'm sorry, I'm an insensative Clod)

        • Re: (Score:3, Insightful)

          by icebraining (1313345)

          "Designing is really a 2 way street, it needs to be done by the user just as much as it does have to be done by the developer, maybe even a little more so on the user."

          But in most small OSS projects the developer *is* the user, at least at design stage. Most developers don't write code to help the community, they write it because they enjoy doing it and/or need it done. Then, they help the community by release the source, when it's too late to make design decisions.

        • by serviscope_minor (664417) on Thursday September 24, 2009 @06:58PM (#29534641) Journal

          Well exactly. Someone up there in the comments mentioned how Open Source Developers like to "Scratch their own itch" - which in my opinion is really the wrong way to tackle a problem.

          What the hell? I have a problem. I choose to fix it. I then offer my solution to the world at large, completely for free. Now you come along and tell me that I've solved my own problem wrong and should have somehow done it so it benefits you more.

          WTF?

          If you want me to work for you, then you have to pay me a lot of money. If you don't like the free itch-scratching stuff I give away, then ignore it ang go about your life as if it was never there.

          Talk about a sense of entitlement...

      • by Monkeedude1212 (1560403) on Thursday September 24, 2009 @05:15PM (#29533509) Journal

        I don't know how many times I've torn my hair out because the 'Help' menu's only item was "About".

        PS - I thought I was the only one. I banged my head on the desk one time and it left a bruise because of that. True story. (I wasn't having a very good day to begin with though)

      • Re: (Score:3, Insightful)

        by selven (1556643)
        Speaking of "About", why is it that this option doesn't tell you anything about the program, but gives you some useless copyright information instead? Can't we call it "Copyright" or something like that?
  • Not the issue.... (Score:5, Insightful)

    by Darkness404 (1287218) on Thursday September 24, 2009 @04:27PM (#29532913)
    The problem is, we have this odd expectation that any software, from a compiler, to a game, to an office suite to a browser should be instinctive by use of other software. That is, they think Word processor == Word. So when you take another word processor such as Open Office, they expect it to work -exactly- like Word. Any differences are seen as "faults". Take someone fully new to computers and have them learn Linux or Windows and chances are they will figure out Linux faster. Take someone who has used Windows all their life and give them Linux they complain because things aren't exactly the same.
    • by jockeys (753885) on Thursday September 24, 2009 @04:36PM (#29533043) Journal

      Take someone fully new to computers and have them learn Linux or Windows and chances are they will figure out Linux faster.

      Citation needed.

      I'm not being a dick, I'm genuinely curious: has there been any study on this topic, beyond anecdotal posturing?

      • by MiniMike (234881) on Thursday September 24, 2009 @05:16PM (#29533515)

        This is easy:

        Step 1- Get two identical rooms. Fill one with computers with your favorite Linux installed, the other with computers with Windows installed.

        Step 2- Put a sign on the door reading "Mac Lab". Use large letters.

        Step 3- Observe resulting behavior. Write paper. Profit!

        (Before anyone gets offended, I think Macs are ok and used them a lot in the past, but think this would be an interesting experiment even though different than the op suggested.)

    • by maccodemonkey (1438585) on Thursday September 24, 2009 @04:42PM (#29533115)
      Would you buy a car that didn't have a steering wheel? Sure, certain software vendors have set certain standards for software interfaces. But the user is king. It doesn't matter who trained the user what to expect, if the user expects something, you should tailor your software to their expectations. If you think it's the users job to learn your interface, the user is just going to keep using Windows because they don't want to spend time learning the Linux way of doing things. Respect your users time.
    • Piffle (Score:3, Interesting)

      by onyxruby (118189)

      You can still perform plenty of validation testing irrespective of what platform a person has used. If a given person can't figure out what your trying to do with the tools provided than the software needs work. When I used to do work for a manufacturer we took people off of the assembly line or equivalent, made sure they knew nothing about computers and used them to perform the testing. If they couldn't figure things out on their own, the test was considered a failure. Blaming the platform or the userbase

    • Re: (Score:3, Interesting)

      by Animaether (411575)

      Take someone fully new to computers and have them learn Linux or Windows and chances are they will figure out Linux faster

      I'm confused.. weren't you talking about word processors just two lines back? Now it's operating systems? What aspects of that operating system, exactly? Are you talking about the desktop management or the CLI?

      That said.. I understand what you're trying to say.. that people are biased from their experience with a 'competing' product.

      On the other hand, that bias may not be a bad thing.

  • by Syncerus (213609) on Thursday September 24, 2009 @04:28PM (#29532927)

    Because so many developers develop Open Source applications for personal satisfaction, they tend to focus on scratching their own itches.

    A characteristic of usability testing is that your goal is to scratch the itch of your customers; your preferences have very little significance in the context of the test.

    It doesn't take a genius to see a potential conflict in the two goals; on the other hand, a developer likes to see his code in actual use by actual human beings. To maximize this use, a developer must at least pay lip service to documentation and UI testing.

    Many developers never make this conceptual leap, however.

  • Shockwave (Score:3, Informative)

    by 2phar (137027) on Thursday September 24, 2009 @04:52PM (#29533275)
    Lack of Shockwave is a big problem with kids.. A lot of childrens' websites feature games that use Shockwave. This was essentially the deal-breaker in setting up an Ubuntu box for a niece of mine recently. Maybe someday these websites will stop relying on it.
  • by PAPPP (546666) on Thursday September 24, 2009 @05:01PM (#29533357) Homepage
    The idea that an interface can be entirely judged by how well a user handles it in the first few minutes of exposure is, in my opinion, one of the bigger *problems* with UI design of late. A quality interface should both be immediately accessible, and SCALE WELL TO MORE ADVANCED USE CASES. In my experience, Gnome, OS X, and the bundled native applications that come with each currently fail miserably at the latter. The former head of Apple's UI team makes a pretty good case for this being a problem here [asktog.com], although the article focuses specifically on a facet of the OS X design philosophy which causes scalability issues, rather than the problem in general. To borrow a line from the article: "The beginner today will be the expert of tomorrow. The user with 200 photos today will be the user with 2000 a year from now. The user with 10 songs today will be the user with 100 songs six months from now. The user with one or two extra apps on the iPhone will be the user with 100 apps three months from now."
  • by Stormwatch (703920) <rodrigogirao&hotmail,com> on Thursday September 24, 2009 @05:09PM (#29533435) Homepage

    True story here: dad's computer had OpenOffice, not MS Office. My sister's experience with OpenOffice's Impress was terrible: she needed to print all slides from a .ppt file, and couldn't find this option. As she had a tight deadline, and I had nearly zero experience with presentation software anyway, I shrugged and installed MS Office. She ran Powerpoint and found her way very easily.

    Just a bit later, I tried to find out how one prints all slides from a presentation.

    Guess what? It's done EXACTLY the same way in Impress and Powerpoint. Same function, same name, same location. See, this is not a "Photoshop versus Gimp" style comparison; interface-wise, they were nearly identical (that was before the "ribbon" thing). If she found her way in Powerpoint, she should have found her way in Impress. Yet, she somehow panicked with the new program.

    What can a developer do about users that won't even TRY?!

  • by melted (227442) on Thursday September 24, 2009 @05:09PM (#29533441) Homepage

    ... as a developer.

    They basically have labs with one-way mirror. User is left alone in a sound-proof room and given a set of tasks to perform. Everything is recorded (including facial expressions and sound), and any developer can take a look at the test either from the adjacent room or from his/her workstation (using Windows Media Player). The only input the user gets is when he gets so confused he can't accomplish the task from the list. In which case the person conducting the test just says "next task" and that's it.

    The experience is really humbling. You just realize that people out there are FAR, FAR less experienced with computers than you thought, and even working their email client is a challenge for most.

    You make your assumptions on the basis of what's convenient for you. Guess what, people out there are not you, and what's good for you is torture for them (the inverse is often true, too).

    We ended up redesigning the entire chunks of the UI sometimes, some features got cut, some scenarios overhauled. And in the end we still didn't do enough of usability testing (IMO), but such is life in commercial software development - you work against an arbitrary schedule.

    • Re: (Score:3, Insightful)

      by gilgongo (57446)

      And in the end we still didn't do enough of usability testing (IMO), but such is life in commercial software development - you work against an arbitrary schedule.

      Lemme let you in to a little MSFT secret here: what you witnessed was eyewash to make you feel better about your job.

      The people that matter at Microsoft know the truth: that when you have a monopoly, all that's needed is to make sure the software doesn't crash on the launch presentation and that it supports as much hardware as possible. Achieve that, and you have achieved your annual bonus because even if MS released a C# horse's butt, billions of corporate slaves would still buy it and sign up for the upgr

      • Um, no (Score:4, Informative)

        by melted (227442) on Thursday September 24, 2009 @08:36PM (#29535523) Homepage

        This was a new product, in a new (for Microsoft) market. We were starting from zero marketshare against firmly entrenched competitors. It took that product about five years to even start breaking even and now it brings in a healthy profit.

        People at MSFT by and large try really hard to put out the best product they can. Unfortunately, in a company the size of Microsoft it's not as straightforward as it perhaps should be. If you work on a product that's already shipped a few versions, you end up having to convince too many people to get anything changed, so unless something is truly horribly broken people tend to pick their fights and argue for the cases where have a greater probability of success.

        Fortunately, this is not an issue with new, v1 products, since you're building from the ground up. Hence, in our case, we've made fairly dramatic changes as we went along.

  • by jsled (11433) on Thursday September 24, 2009 @05:51PM (#29533969) Homepage

    Go do volunteer basic computer literacy session for your local senior center. Don't try to convert them to linux or get them using Firefox or anything dumb like that. Just ask what their problems are, and how you can help. You will quickly understand how broken and unintuitive computer software is.

  • by lennier (44736) on Thursday September 24, 2009 @06:02PM (#29534083) Homepage

    this is the same Mark Shuttleworth who removed update-notifier and then when hundreds of beta-testers said 'please put that back' on the infamous https://bugs.launchpad.net/ubuntu/+source/update-notifier/+bug/332945 [slashdot.org] he personally said 'no, I'm not listening to you'.

    He said it politely:

    "I'm marking the bug wontfix on the basis that we are confident the behaviour as at 9.04 release is a good one. I wouldn't be surprised for the conversation to continue though I do ask that it continue in a good spirit. If significant data shows this to be a suboptimal choice in future, we will revisit the point, but for now the question is settled."

    but it was still a WONTFIX in the face of overwhelming public opinion to the contrary.

    I'll believe he listens to users when he actually listens to the users.

    • But he was right... (Score:5, Informative)

      by imtheguru (625011) on Thursday September 24, 2009 @08:10PM (#29535361)
      But Mark's observation was right for bug 332945 [launchpad.net]!

      For the benefit of those not familiar with this... the old behaviour of displaying updates was to display an icon next to the clock. The new behaviour is:

      • When there are security updates, Update Manager will open and show them (plus any other available updates) within a day.
      • When there are non-security updates, Update Manager will open and show them *one week* after it was last opened (whether it was last opened manually or automatically, and regardless of whether updates were actually installed then).
      • When there are no available updates, Update Manager will not open automatically at all.

      Friends' Ubuntu installations were rarely updated due to the limited attention received by the little icon. With the new [minimised] update window, the machines updated weekly.

      It all comes down to visibility.

      Cheers.

    • by ZorbaTHut (126196) on Thursday September 24, 2009 @08:17PM (#29535407) Homepage

      You're confusing two very different things. "Pay attention to the user's behavior" and "listen to what the user asks for".

      The first is always valuable. Seeing what users do is just plain good. You should be doing that. You should absolutely be doing that.

      The second, however, is a frequent mistake. Users don't know what they want. They know what they want to do, and they either know they can't do it or they know how they used to be able to do it, but the ideas they come up with to fix that issue tend to range the gamut from "barely acceptable" to "horrible".

      Any change you make to an existing UI - *any change whatsoever* - will result in a storm of people calling for blood. No matter how good the idea is, no matter how good the change is, people will scream for it to be changed back. If you want to create a good UI, at some point you just have to ignore this. People yell for reversion, you tell them "no", and a few months down the line you find out if you made the right call or not.

      You might think he made the wrong decision here, but "listening to the users" has absolutely nothing to do with real user experience testing.

    • by Dhalka226 (559740) on Thursday September 24, 2009 @08:33PM (#29535501)

      The two really aren't the same thing, they only seem that way because you've erroneously over-simplified his position to "listen to the users."

      User experience testing is essentially about usability. If you put some dude who has never seen your software in front of it, can he use it to get his work done? Is there anything seriously impeding his ability to 1) learn or 2) use the software?

      What you're referencing is that something changed and people don't like the change. For starters, most people don't like change even if it is ultimately change for the better. More to the point though, it has nothing to do with learning or using a piece of software. They simply preferred one behavior to another for a set of reasons that may or may not address any of the reasons the change was ultimately made. A user below suggested that the previous situation (apparently, an icon in the dock for updates) was terribly ineffective but that the new system now achieves much higher update rates. In a situation like that, where some users are annoyed by a behavior but there is a demonstrable and measurable net positive to the change, reverting it is probably the wrong answer even if his motto was "listen to the users."

      For what it's worth, as somebody who has no vested interest in the change either way I think his response was perfectly reasonable.

      but it was still a WONTFIX in the face of overwhelming public opinion to the contrary.

      Was it overwhelming public opinion? Sorry if I'm wrong in my assumptions, but I smell some bias in your post. It seems to me like you were one of the ones who want the change reverted. There's nothing wrong with that, but combine selection bias with the general megaphone that negative reactions get compared to positive ones (far more people hop on to review something they hated than loved) and I don't know it's as clear-cut as you suggest. Plus, this is a bugtracker. For all the increased likelihood of bad comments to good in general, most people wouldn't even think to log onto a bug tracker if they liked or accepted the new behavior. And why should they?

      It's also worth mentioning that "listen to your users" wouldn't necessarily equate with "give your users everything they want" as well.

I tell them to turn to the study of mathematics, for it is only there that they might escape the lusts of the flesh. -- Thomas Mann, "The Magic Mountain"

Working...