An App to Boil Down Online User Reviews 82
An anonymous reader writes "Is this a glimpse at the future of the Semantic Web? A new startup named Pluribo has developed a technology that can auto-summarize user reviews on the internet. It is a Firefox extension that can take a webpage filled with reviews and condense it down into a couple of sentences. Currently, it just works with Amazon electronics, but the potential seems incredible. Ars Technica took an in-depth look."
Quick. (Score:5, Funny)
Methinks it would make it worse (Score:4, Interesting)
Since they explicitly mention Amazon, heh, my experience with Amazon's user reviews has been pretty bad to start with. Caveat: it's not about electronics, but I do buy games and the occasional DVD movie off Amazon.
My impression is that the amount of fanboyism, astroturfing and bullshit is... epic. Monumental.
E.g., read some reviews for a game that's not released yet. My favourite example was Gothic 3, when it wasn't even in beta yet, or even alpha. The only thing anyone had were some screenshots of what the graphics engine can do. That's it. Nobody had anything playable yet, probably not even the devs.
Well, people were already writing reviews in which it's the greatest game ever, and the gameplay rules, the graphics are the best since Michelangelo, etc.
When released, the game was a buggy mess that didn't even vaguely resemble those "reviews". The graphics had some major glitches. Quests could be broken because the NPC had fucked off, and I know someone who encountered that right in the freaking intro. The game had a nasty memory leak, where eventually it would start to barely crawl and eventually crash... often while saving, leaving you with a corrupt and unusable saved game. Gameplay too was a broken fuckup: e.g., combat was a broken whoever-hit-first-wins affair, because then the other would be continuously interrupted and unable to hit back or change weapons or whatever. Even a flea could probably kill you, if it hit first. Etc.
Most of that stuff _still_ hasn't been fixed, after more than a dozen patches and the publisher giving up on it.
But, of course, going by the user reviews, you'd think it's the greatest game ever.
Now as a human, you can filter out the blatant bullshit, see which reviewers better reflect your taste and didn't post too much bullshit before, etc. I'm skeptical that a program can be too good at doing the same.
But I have an even worse fear: that once people figure out that they only need to game a program, and how, we'll see even more fanboyism, astroturfing and bullshit. Plus an army of sock-puppets to mod each other up, if the bot takes that into account. Basically, think about all the link farms and link spam on the net to game Google's page rank. Now think the same for a bot aggregating reviews. I find that scary.
So, no, I don't want it on Slashdot too. Basically, would you really want 300 goatse links, just so the bot includes it in the digested version?
Re: (Score:2)
It's a good idea, but just needs to be under your control. At that you could have it apply negative weights to votes from people who voted those articles up. They'd then push those down, out of your way.
This is only less than useful because it'll be a one-size-fits-all solution and, if YouTube comments are any gauge, most people aren't as discerning as you.
P.S. Yes, Gothic 3 stunk. Unplayable at release, barely playable weeks later with patches. 2+ minute load times (quick-load, hah!) on modern hardware, co
First Post (Score:5, Funny)
I for one welcome our hot grits pouring overlords.
Imagine a Beowulf cluster of Al Gores in Soviet Russia releasing Duke Nukem Forever, you insensitive clod!
In Korea, only old Natalie Portman must be new here.
All your Linux are belong to us Sharks with frickin' laser beams attached to our heads.
1)Stephen King is dead
2)BSD is dying
3)Profit!
There. Fixed that for you.
Re: (Score:1)
1)Stephen King is dead
2)BSD is dying
3)???
4)Profit!
There. Fixed that for you.
Re: (Score:1)
Imagine a Beowulf cluster of Al Gores in Soviet Russia releasing Duke Nukem Forever, you insensitive clod!
Yes, but does it run Linux?
:)
(Sorry)
Here, let me test (Score:4, Funny)
Enter> The Wicker Man, 2006
Result: "Sucks monkey balls"
Hey, it really does work!
Re: (Score:2)
Enter> Anything with Nicolas Cage
Result: "Sucks monkey balls"
Hey, works on more general queries too!
How about a comment synopsis generator (Score:2)
Can we apply this to Slashdot comments? Please? Please? Pretty Please??
I'd love to get a one page summary of all the informative, insightful and interesting comments.
Re: (Score:2)
I tried that. See 'first post' above. It got modded flamebait.
Re:How about a comment synopsis generator (Score:5, Funny)
I'd love to get a one page summary of all the informative, insightful and interesting comments.
[url=http://slashdot.org/~Rob+Kaper]Here you go[/url].
Re:How about a comment synopsis generator (Score:4, Funny)
Joke's on me, this time, I guess. *sigh*
Re: (Score:3, Funny)
You could just browse at 4 and apply an extra -2 modifier to all Funny comments.
Re: (Score:1)
Re:How about a comment synopsis generator (Score:5, Informative)
Man, you guys really don't know about Alterslash [alterslash.org] yet?
Re: (Score:1)
One problem (Score:5, Informative)
The application seems to assume that the best summary is the one with the most correlation to the other posts, in other words: the most common viewpoint. While that may work fine for user reviews, in most cases the viewpoint of the masses is usually not the best.
Re: (Score:2)
But seeing as though it is specifically targeted at user reviews, and currently just Amazon Electronic ones, it would seem to be a great way to summarize the lot.
I mean, what do you do when you're looking at user reviews of products? Look for what the common threads are, or focus on a few outliers?
Re:One problem (Score:5, Insightful)
...or focus on a few outliers?
Yes, probably more than half the time actually, because 90% of the reviews are about as accurate as Slashdot comments, or as accurate as that percentage I just made up.
When it comes to hardware, I may work a bit better, however with the diversity of hardware that exists, that one comment that you missed saying "but don't EVER purchase this and install it if you have an [Insert Product]!" might be exactly what you need to know and the rest is just fluff. The same goes for a lot of software, and you might miss out on that "great find" by that one guy that said "its ok, but it's less efficient/configurable/easy and more glitchy/resource hog than [Product X]"
Re: (Score:2)
Actually, I really just
Comment removed (Score:4, Insightful)
Re:One problem (Score:5, Insightful)
Precisely: I like reading about the edge cases, because Joe Random is a freaking moron when it comes to electronics.
The whole "me too" effect is a huge part of today's marketing. That's why everyone and their mother wants a freaking iPod/iPhone.
Re:One problem (Score:4, Interesting)
Re: (Score:1)
Re: (Score:1)
I approved it. (Score:4, Informative)
I approvied the Pluribo extension for being pushed to public. All I can say is that it gets most of its data from the Pluribo server, and does very little client-cide besides display the data. At the moment it is *extremely* limited, literally to a handful of products. Apparently their system doesn't do the scanning of comments automatically otherwise it would work for everything, whereas it actually has to query the Pluribo server to get results.
Re: (Score:3, Insightful)
Re: (Score:1)
Pft... wake me up when it can play free music [mozilla.org].
I have no shame.
Okay, now here's a request: (Score:4, Interesting)
...is there any way to have it filter out the obvious astroturfers and trolls?
Seriously, any big-name product or service will have a coterie of fanboys (or paid astroturfers) who will praise something no matter what, and a flock of trolls who will point out everything wrong with it, no matter what.
Do that, and it'd be one hell of an advancement in filtering. :)
Re: (Score:1)
It might do that, even with Slashdot. (Score:1)
Astroturf is repetitive, so it will get listed in one place. You won't be able to eliminate the talking points but you won't have to read them 50 times to find one genuine opinion. Both will be weighted as original content. You will then have to use your brain as you do now. At the same time, you can eliminate distractions like "hot grits". It took me a while to realize this but now I see that it is true.
Re:Okay, now here's a request: (Score:5, Interesting)
I was going to post something similar. I was apartment shopping earlier this year, and the amount of astroturfing and astrotrolling* was incredible.
Any filter that decreases the amount of information that I can use to evaluate the "truthiness" of a review is a bad thing. What's more, if filters like this catch on, people will be selling FEO (filter engine optimization) services to game the filters with their astroturf, and then the reviews will become completely useless.
* In case I just made up a word, what I mean by "astrotrolling" is people who post shit about a product to get people not to buy it because they have a separate axe to grind against the seller. In the case of apartments, it's often poor tenants who tore up the apartment/broke the lease/got evicted and still amazingly expected their security back.
Re: (Score:3, Insightful)
That is what this article seems to me. It seems that people don't want to invest the time to actually learn or resear
Re: (Score:1)
Err, what does online reviews filtering have to do with reading a book?
Hell, I pretty much do this unconsciously anyway when evaluating something, for things that I am familiar with (e.g. computer parts, electronics, etc).
Also - last I checked, Powell's Books hasn't gone out of business or become a glorified magazine stand. Folks on the train haven't stopped reading books and novels (of an amazing variety judging by last night's commute home).
It's a question of priorities - I have zero problems with curling
Birth of a new word: "astrotrolling" (Score:1)
astroturfers and trolls (Score:4, Interesting)
I run a site that catalogs free books, and accepts user-submitted reviews. (See my sig.) It's a constant source of amazement to me what a low level of morality (and intelligence) some authors have. They'll add their book to the catalog even though it's not free. (The site's UI tells them very clearly that it has to be free online in order to be listed.) Then they'll post their own "review" of the book, which reads exactly like a dust-cover blurb rather than a review. Then I check the email address they used to sign up on the site, and it's the same as the email address of the author of the book -- this despite the fact that the button they had to click on to submit their review was labeled I am not the author, and have no personal, professional, or business relationship with the author. I am submitting my review..
About 50% of the reviews I get are like this, and I have to delete them by hand. I don't actually get that many reviews submitted, which is a good thing in a way, because if the site was really busy I'd never be able to keep up.
I don't think there's any way of solving this problem, since the internet was designed for anonymous use, and even if it was technically feasible to verify identities on the internet, I wouldn't want to do it. Amazon tries fairly hard to deal with this problem. These days they won't let you submit reviews unless you've bought something from them, which is probably a reasonable way to stop sock puppets. They also try to get you to build up a reputation for your online persona, even if it's not publicly tied to a meatspace identity. That doesn't really work that well, though. For instance, there are certain people on amazon who submit something like ten reviews per day, 365 days per year -- obviously they're not really reading all those books. I also don't see any way to stop the phenomenon of the author getting his friends, family, and grad students to write good amazon reviews of his book.
Because of all this, I'm suspicious of any statistical method of analyzing user-submitted reviews. You just have no way of knowing which reviewers are honest. You really have to look at the individual reviews and see if what they say makes sense. Ebay feedback is an example of how silly this can all get, even in a community where people really do have long-term online identities that they have an interest in maintaining good reputations for. What the heck does it tell you if the seller has 99% positive feedback? Absolutely nothing. You have to read the 1% negative reviews and try to evaluate whether they sound reasonable.
A reflection of the times. (Score:5, Insightful)
This is a serious reflection of our current times, where people's eyes gloss over if the concept at hand is not condensed into a convenient sound-byte.
I suppose you could call it the bleeding edge where complacency meets the loss of freedom and the fall of darkness where critical thought once stood.
Now there is enough probable demand to launch a startup designed to remove what minimal labor people are interested in dedicating to the quality of even their leisure time.
I'm sure many fantasize about strangling people this lazy/complacent, but honestly if they're unconscious enough not to care about their own toys, do they really possess a "life" for you to take from them?
Err, which times? (Score:1)
Dude, even back in history the sound bite, even if it makes no sense nowadays ("Tippecanoe and Tyler Too!") has been prevalent.
Also, "our current times" include things that simply didn't exist before: near-universal literacy, 24/7/365 media, and a desire to do something with one's time off other than simply sit around in a half-drunken stupor somewhere while catching up from the work-week's exhaustion. The latter part was pretty much what the vast majority of humanity did with what little leisure time they
24/7/365?? (Score:2)
Do you mean 24/7/52? Or perhaps 60/24/365? But 24/7/365 seems a bit redundant?!?
I bet I made a mistake in this post, tis the law of slashdot!
Re: (Score:1, Redundant)
86400/1440/24/7/365.2425/52?
Re: (Score:2)
Also, "our current times" include things that simply didn't exist before: near-universal literacy, 24/7/365 media, and a desire to do something with one's time off other than simply sit around in a half-drunken stupor somewhere while catching up from the work-week's exhaustion. The latter part was pretty much what the vast majority of humanity did with what little leisure time they got.
ah, so the world sucked worse then for joe everyman (so we should just bend over and take it is that right)?
Re: (Score:1)
Who says you have to? It can be a switch, an option.
I'm to lazy to read TFA (Score:5, Funny)
Re: (Score:2)
Good point, I'm surprised Slashdot doesn't do that for us. Here ya go:
"A new startup named Pluribo has developed a technology that can auto-summarize user reviews on the internet. It is a Firefox extension that can take a webpage filled with reviews and condense it down into a couple of sentences."
Re:I'm too lazy to read the summary (Score:4, Funny)
Could you boil it down to a word?
Forty-two
Re: (Score:2)
!
It's not summarization. (Score:4, Interesting)
It's heavily templatized generation of language based on the automatically extracted sentiment data. The important difference here is that the language of the summary does not include phrases from the original user reviews. While this is a new twist on the old problem, automatic extraction of evaluation criteria and sentiment analysis in product reviews are not new. Heck, even Microsoft has a working system for that (electronics only):
http://search.live.com/products/?q=nuvi%20350%20GPS%20-%20Asian%20American%20(City%2FVehicle%2C%203.5%22%20LCD)&p1=%5BCommerceService+scenario%3D%22reviews%22+docid%3D%222BECBBF6F17C98618C2E%22+p%3D%2220df8fe62a9b4e9490993ff7b91032af%22%5D&wf=Commerce&FORM=ENCA [live.com]
See the bars on the left, and be sure to click through to the individual sentences. It's spooky how accurate that thing seems to be.
The problem with all these systems is that they're heavily domain dependent. You will use different language to write a review of a book than for kitchen appliance. In fact, you may even use different language from different kinds of books or different kinds of kitchen appliances. Worse yet, some things are notoriously difficult to accurately measure sentiment on. Once innuendo and sarcasm become frequent, all hope is lost - you need strong AI to figure that out.
This is not to say these systems are useless - to the contrary, they are very useful in their respective domains. This is just to say that the only new thing I see here is the generated blurb.
Re: (Score:3, Funny)
I think that Microsoft live product already has strong AI; it knows how to brown-nose so Microsoft doesn't kill it:
"Windows? Millennium Edition, or Windows Me, is the home operating system for PC?s that brings the richness and convenience of the digital world to your home. Windows Me is designed specifically for the home PC user. It represents the first major milestone towards advancing the vision of the Windows Division and further simplifying the computing experience for consumers. Windows Me delivers in
Re: (Score:2)
Worse yet, some things are notoriously difficult to accurately measure sentiment on. Once innuendo and sarcasm become frequent, all hope is lost - you need strong AI to figure that out.
that's why Amazon is easy, you have the stars rating to go by. i think they're just marrying the fact that you mention a particular aspect of the product to your overall rating of the product, and in the end they can make a nice graph saying what the average rating is for a review that mentions a particular aspect.
Mac OS X Summary Service (Score:4, Informative)
Built into Mac OS X is a Summarize Service (click on the application's menu and point to "Services" and choose Summarize) that is pretty cool. Unfortunately, Firefox was never coded to work with that OS X native service, so I had to copy and paste the text of that Ars Technica article into Text Edit and then use that service. But here is the resulting summary (I had to do as short a summary as possible):
Ugh. The opposite of what I want. (Score:5, Insightful)
This sounds like the opposite of how to get useful information out of reviews, and more like the "consumer products" equivalent to the automatic resume scanner.
You know the resume scanners I'm talking about -- the ones that circular-file the candidate who took three years off work to get his Ph.D. in cognitive science (and whose thesis is a perfect fit for your business plan), preferring, instead, the guy who listed "20 years PROLOG, PL/1, BASIC, C, 10 years C++, 5 years Java, MCSE, A+", because obviously the second guy triggers more buzzwords. Because the HR drone won't understand any of the resumes, he/she just picks whichever one the scanner selects, and that's typically the one with the fewest career gaps and the most buzzwords. ("But that other Ph.D guy only has one or two languages, this guy has six! And that Ph.D guy's been out of work for three years, so obviously nobody would hire him!")
Ten reviews reading "Works. Fast, cheap, lightweight" and three reviews reading "Doesn't work" don't tell me anything, other than that the product might have reliability issues.
One review reading "Didn't work the first time. The manual doesn't mention that you have to make sure the jumper is in the correct position first, and then it works. I own an XYZ-123 and this new product was at least as fast, but at about half the price. Weighs about a pound." tells me everything I need to know -- that the three people who claimed it didn't work almost certainly didn't know how to configure it correctly, and that the first seven reviewers never had a problem because they weren't part of the edge case.
Re: (Score:2)
This is not the Semantic Web (Score:2, Informative)
Re: (Score:1, Insightful)
The idea behind the real world is that content providers will put bullshit in their tags, so that your hopelessly naive tools will instead create a fauxtlogy so that programs can easily duped into misleading customers to purchase worthless products.
Re: (Score:1)
Keywords in Context (Score:3, Interesting)
I'm concerned how well the applet properly discerns the meaning of words in context. For example, just because I mention that a product is 'a portable laptop' does not mean I am impressed with it's size or weight... it's just the category the product falls in. But judging from the screenshots in the article, this exact error was committed by the plugin.
Reading natural language is hard, and I'm of the opinion that a Firefox plugin just won't cut it for understanding the nuanced opinions given by reviewers.
Re: (Score:2)
Reading natural language is hard, and I'm of the opinion that a Firefox plugin just won't cut it for understanding the nuanced opinions given by reviewers.
now toss it some ebonics or wow-ese and watch it go nuts..
Re: (Score:1)
fetchSummary: function(doc) {
var plid = pluribo.extractPlid(doc);
if (plid) {
var req = new XMLHttp
Spam reviews... (Score:1)
Unfortunately, the product will never be able to filter out useless spam reviews that artificially inflate product ratings, such as "OMFG THIS IS THE BEST!!!!" (entire review) (especially for products that aren't even released yet) or paid site-fillers like "this is good I'm so glad [this site] made it available".
I expect most media product reviews will summarize to something like "This is the best ever, if that's your kind of thing."
Re: (Score:1)
Most of us have already figured this out... (Score:1)
The way to properly get information from online reviews is to sort from lowest rating to highest, skip the trolls, and read until you get to the fanbois.
Re: (Score:2)
MacOS X: How to Summarize the Contents of Document (Score:4, Interesting)
This document explains how to use the Summarize services available in Mac OS X applications.
If you have a long document, you can use the Summarize service to get a summary of the contents. For example, use this to get a short version of a long page on a Web site.
To get a summary of a document, select the text and choose Services from the application's menu, then choose Summarize.
If the application you are using doesn't support services, copy the text to a TextEdit document to get a summary.
Note: The information is this document comes from Mac OS X Help, the help system included with your computer. It is based on Mac OS X 10.1.2. If a different version is installed on your computer, choose Mac Help from the Help menu. Updated and expanded information may also be available in other Knowledge Base documents.
http://docs.info.apple.com/article.html?artnum=61336 [apple.com]
(but i know this feature was in OS 9 or earlier)
love the idea of this... (Score:1)
Would be very nice if it could process the reviews on Newegg. Some items have tons of reviews.
Soon to be banned (Score:2)
By people who think it's slimy for a program to manually scan a lot of pages you aren't going to read anyway.
Several summarizer tools already there (Score:3, Informative)
State of research in the area (Score:1, Interesting)
NIST runs yearly evaluations regarding automatic summarization. Some information about that stuff is available at http://www.nist.gov/tac/ (used to be http://duc.nist.gov/).
There are two main approaches: the domain-dependent template filling plus text generation, or the domain independent statistical sentence extraction. And either way, the quality of the generated summaries is far far away from what a human can write.
While machine-learning-powered research systems are much better than Word or OSX summariza
whatcouldpossiblygowrong, perfecttrap (Score:1)
someone is summarizing for me according to his algorithm which i have no clue about as an end user.
Useless Chaff (Score:2)