Does Software Need a Siskel and Ebert? 169
theodp writes "Over at Scripting News, Dave Winer laments the lack of serious software reviews in the NY Times. That wasn't always the case, recalls Dave. 'When they started doing software reviews in the early '80s it was with the usual Times flair,' says Winer. 'But somewhere along the line they stopped taking tech seriously. It's as if they would only review Saturday morning television shows. How could television like The Sopranos or Breaking Bad take root in the culture if there was no criticism that discussed it? Yet that's where we are today with software.' So, does software need a Siskel and Ebert (or A.O. Scott and Manohla Dargis for you highfalutin NYT readers!)?"
Software is too plentiful (Score:3, Interesting)
Mac Word 6.0 by David Pogue (Score:5, Interesting)
I emailed him and told him I was disappointed.
Re:Won't work with FOSS. (Score:5, Interesting)
I think you're trolling, but as a FOSS advocate myself, I wish you were wrong.
It's much like the problem of racism. After an advocate sees enough incidents of racism, every decision they don't like is suspected of being racial discrimination. Similarly, the myth that "you get what you pay for" is so pervasive that FOSS is often discriminated against, and there's a lot of money aimed at keeping it that way.
FOSS advocates like myself often suspect a bias in bad reviews, partly because we've seen companies like Microsoft pay their shills to bash FOSS, and partly because even honest reviewers don't have any investment in the software they get for free. They'll often dismiss it at the slightest problem when a paid-for product would get a second chance. There's also the familiarity bias, where the latest version of a program will be rated highly because the reviewer's already familiar with older versions, but an alternative has slight differences that the reviewer doesn't understand. While the two packages may be equal to a new user, the reviewer will rate the one they're most familiar with higher. Since FOSS usually has a minority market share, this bias is often against it.
The best way to avoid the rabid hordes of FOSS advocates is to have a professional writing style. Before writing any reviews, show a history of technical knowledge and a willingness to thoroughly examine everything new. In the reviews themselves, explain where and how you got the software (disclosing any conflicts of interest), and preferably also document how much prior experience you have with that program's other versions and competitors. In short, show us that you acknowledge your own faults.
Re:We need reliable reviews (Score:0, Interesting)
"VG" = "videogame"?
That was too much to type, at least once?
Okay, I'd like to call for a vote on the prohibition of new, unnecessary acronyms. If it isn't already listed here, it's disallowed.
http://www.allacronyms.com/tag/usenet [allacronyms.com]
It's all we had, and we got along fine. Even walking to school uphill in the snow, both ways.
"SMH". ;)
You can't do real software reviews (Score:4, Interesting)
It's absolutely impossible to do real software reviews of many software products without risking getting sued. This is due to the industry using NDA's for software that prohibit unapproved reviews. NDA's are why on release day you will all of a sudden see a plethora of reviews on release day. Reviews off of sites like Amazon are largely worthless due to the sheer number of shills and the most popular reviewers getting large quantities of merchandise for free.
One merely needs to look at what happens with video games to know why. If you work for a video game magazine and give a scathing negative review you won't get selected to review the next product from that publisher. After a while you end up being unemployable as video game reviews have to be ready for release day. It doesn't take long to realize you have to carefully write about a game without pissing off the publishers. The net result is that pretty much every game review web site effectively becomes a shill for the publishers as they can't afford to miss out on day zero releases.
Take your favorite site and select all their reviews and put them on a bell curve. Most (average) software should fall somewhere in the middle of their scale. In practice you will find many sites will give average reviews of a 7 or 8 on a 10 point scale. An honest site will fit the bell curve, a dishonest site will quickly be exposed by the bell curve distribution being shifted towards better scores. These problems are why some sites make claims about refusing to sign NDA's, they are showing that they have more integrity to give honest reviews.
This can even extend through to things like operating systems where many beta or rtm releases have excluded the right to review the product without approval in exchange for getting an early release. One simply needs to review the history of Operating System releases to see the effect of reviewers that are afraid to piss up companies. Look back at Windows Me, Vista, Mac OS's before 10 and so on and you can find a plethora of initial approving reviews (ZD Net in particular comes to mind).
The problem gets even worse with actual commercial software. Read your fine print from Oracle or any other commercial product and you will almost certainly find the license prohibits benchmarking and other similar activities that could be used for a review - especially for trial versions. In addition to license issues the cost for commercial software makes it unfeasible to purchase.
Trying to review enterprise class software becomes even more unfeasible as you can't simply install it. In order to properly set it up you need a consultant who knows the product fairly well and that is cost prohibitive for a company that isn't even going to use it. Since enterprise software tends to include language in the EULA that prohibits unapproved reviews no consultant, who naturally depends on having a good relationship with the publisher, is going to help you if you might say critical things about it.
So how do you get a real review of a product that your considering investing a lot of money in? Go to a conference or users group for the software, find an admin who's been using it and take them out to a nice dinner for an off the record review of how the product actually works.
Re:We need reliable reviews (Score:2, Interesting)
Well, which software needs to be reviewed? If you're talking about the apps in Microsoft Office, then you compare them to...what? the various open-source alternatives? OK. Well, that review is going to be inherently biased depending on the review's default audience. The Open-source (read: Linux) world already hates Office and Microsoft. the non-Linux environment really has no viable alternative to compare it against, as all of its past competitors in that space have fallen away (WordPerfect, Lotus, et al). You may have your business, and decide to use Libre Office. Great for you and your 2 employees. A Fortune 500 company with 50,000+ employees moving off of Office? Not going to happen.
Then, you get into case studies. of course using Microsoft Word for publishing & layout is full of fail. it's been that way for more than 20 years. Word users barely grasp the concept of styles, so they continually just manually format everything, including using spaces and tabs to layout text. MS doesn't make the ruler bar visible by default anymore in Word. They long ago obfuscated the menus that modified the basic structure of a document (section breaks, etc. The commands are still there, but you'll need to find them and customize the ribbon-bar for yourself).
As much as Microsoft Access gets scoffed at, at least it's still true to what it does. At its core still works and is used the same way it always has been, if only through some relatively benign neglect. And unless you're truly a database geek, all the talk about database concepts is...geekery. And the database pros in the company will be able to work with it to upscale things, even if they have to plug their noses to do it (unlike FileMaker Pro).
So, what else, then? Web browsers? OK, there isn't really an "application" one can use with a web browser, so you're left expounding on vapid "experiences", really. If you want to use something that works in Chrome but not IE, Safari or Mozilla, then fine. If you need to use things that only work through IE (e.g., SonicWall firewall remote connections...), then, well, IE "wins", right? Or do you review Angry Birds compatibility in web browsers (chrome.angrybirds.com)?
Then look at Ed Bot's recent rant against GMail... that does not advertise for this space eitiher.
What else is there to compare, then? Evernote? Um. OK. So, then, which competitors do you compare it against? iOS only? android? both + Windows?
So about the only place left where semi-objective reviews seem to be done are for enterprisy vertical apps. Which is "better", SAP, PeopleSoft, Oracle Financials, Microsoft Dynamics?
But these are difficult to do. InfoWorld used to try and do these back when, including rating installation and setup of the different platforms, configuring the app for some imagined workload and business configuration, then running some basic functionality and rating the ability to do some customizations. At some point, the reviewer has to rely on the company's sales engineers to do a bunch of stuff.
Those reviews are/were done in the appropriate places, but again, they're for CTOs etc to look at, really. The end-user is going to have relatively equally sucky experiences with all of them.
Does the NYT review wrenches? (Score:4, Interesting)
With the exception of entertainment and the rare 'culturally relevant' application, the vast majority of software is primarily a tool to get its job done, rather than an item of artistic merit in its own right. The New York Times reviews are — for the most part — cultural reviews; they're not the appropriate venue for most software reviews.
With that said, there are those exceptions where one can speak about the artistic or cultural merits of a piece of software, and my strong impression is that the Times has never really stopped speaking about those. The difference between the '80s and today is that at that point, there was so much less understood and so much more that was new in the world of software that a lot of what came out was of cultural relevance and worth talking about on those merits.