An App to Boil Down Online User Reviews

An anonymous reader writes "Is this a glimpse at the future of the Semantic Web? A new startup named Pluribo has developed a technology that can auto-summarize user reviews on the internet. It is a Firefox extension that can take a webpage filled with reviews and condense it down into a couple of sentences. Currently, it just works with Amazon electronics, but the potential seems incredible. Ars Technica took an in-depth look."

Quick.

Somebody fix this so it work's on /. Maybe then I'll RTFA.

First Post

I for one welcome our hot grits pouring overlords.

Imagine a Beowulf cluster of Al Gores in Soviet Russia releasing Duke Nukem Forever, you insensitive clod!

In Korea, only old Natalie Portman must be new here.

All your Linux are belong to us Sharks with frickin' laser beams attached to our heads.

1)Stephen King is dead

2)BSD is dying

3)Profit!

There. Fixed that for you.

Here, let me test

Enter> The Wicker Man, 2006

Result: "Sucks monkey balls"

Hey, it really does work!

One problem

The application seems to assume that the best summary is the one with the most correlation to the other posts, in other words: the most common viewpoint. While that may work fine for user reviews, in most cases the viewpoint of the masses is usually not the best.

Re:One problem

Not to mention, the truth is in the details. A lot of reviews are from people that don't know what they're talking about (computers related reviews) or obviously put there by the company that owns the product (hotel reviews, web hosting reviews, etc).

This is interesting, but it won't replace actually researching something to find out what others really think about it.

Re:One problem

Precisely: I like reading about the edge cases, because Joe Random is a freaking moron when it comes to electronics.

The whole "me too" effect is a huge part of today's marketing. That's why everyone and their mother wants a freaking iPod/iPhone.

Re:One problem

That is why geology geeks use different phones [amazon.com] that may not be all fancy and la dee da but they can be dropped from moving vehicles, be submerged in water/beer and still work to call into town for more supplies/beer. I do not understand the fascination with cell phone internet access. The majority of use I see for it is people in bars looking up statistics to win arguments or endless texting and picture sending from giggling girls. I have a GPS navigation system in the car as well as a laptop for full blown internet access when I need it. 95% of the time I have no desire or reason to be online when I am away from home as I have 10's of thousands of textbooks and reference books that I can use on my ebook reader. Thank you cheap SD cards.

Re:One problem

...or focus on a few outliers?

Yes, probably more than half the time actually, because 90% of the reviews are about as accurate as Slashdot comments, or as accurate as that percentage I just made up.

When it comes to hardware, I may work a bit better, however with the diversity of hardware that exists, that one comment that you missed saying "but don't EVER purchase this and install it if you have an [Insert Product]!" might be exactly what you need to know and the rest is just fluff. The same goes for a lot of software, and you might miss out on that "great find" by that one guy that said "its ok, but it's less efficient/configurable/easy and more glitchy/resource hog than [Product X]"

I approved it.

I approvied the Pluribo extension for being pushed to public. All I can say is that it gets most of its data from the Pluribo server, and does very little client-cide besides display the data. At the moment it is *extremely* limited, literally to a handful of products. Apparently their system doesn't do the scanning of comments automatically otherwise it would work for everything, whereas it actually has to query the Pluribo server to get results.

Re:

Well, you're not going to download a thousand product reviews in a couple of seconds, let alone summarize them. On the other hand, I would love to have this kind of functionality available on my own website, perhaps through a drupal module - a comment summary block would be dandy. Heck, you could even have summaries by tag - just summarize all the content tagged with "sco" for example, and find out that they are litigious bastards and that their unix sucked. I wonder if anyone is working on a Free/Open engi

Okay, now here's a request:

...is there any way to have it filter out the obvious astroturfers and trolls?

Seriously, any big-name product or service will have a coterie of fanboys (or paid astroturfers) who will praise something no matter what, and a flock of trolls who will point out everything wrong with it, no matter what.

...now how do you filter those out?

Do that, and it'd be one hell of an advancement in filtering. :)

/P

Re:Okay, now here's a request:

I was going to post something similar. I was apartment shopping earlier this year, and the amount of astroturfing and astrotrolling* was incredible.

Any filter that decreases the amount of information that I can use to evaluate the "truthiness" of a review is a bad thing. What's more, if filters like this catch on, people will be selling FEO (filter engine optimization) services to game the filters with their astroturf, and then the reviews will become completely useless.

* In case I just made up a word, what I mean by "astrotrolling" is people who post shit about a product to get people not to buy it because they have a separate axe to grind against the seller. In the case of apartments, it's often poor tenants who tore up the apartment/broke the lease/got evicted and still amazingly expected their security back.

astroturfers and trolls

I run a site that catalogs free books, and accepts user-submitted reviews. (See my sig.) It's a constant source of amazement to me what a low level of morality (and intelligence) some authors have. They'll add their book to the catalog even though it's not free. (The site's UI tells them very clearly that it has to be free online in order to be listed.) Then they'll post their own "review" of the book, which reads exactly like a dust-cover blurb rather than a review. Then I check the email address they used to sign up on the site, and it's the same as the email address of the author of the book -- this despite the fact that the button they had to click on to submit their review was labeled I am not the author, and have no personal, professional, or business relationship with the author. I am submitting my review..

About 50% of the reviews I get are like this, and I have to delete them by hand. I don't actually get that many reviews submitted, which is a good thing in a way, because if the site was really busy I'd never be able to keep up.

I don't think there's any way of solving this problem, since the internet was designed for anonymous use, and even if it was technically feasible to verify identities on the internet, I wouldn't want to do it. Amazon tries fairly hard to deal with this problem. These days they won't let you submit reviews unless you've bought something from them, which is probably a reasonable way to stop sock puppets. They also try to get you to build up a reputation for your online persona, even if it's not publicly tied to a meatspace identity. That doesn't really work that well, though. For instance, there are certain people on amazon who submit something like ten reviews per day, 365 days per year -- obviously they're not really reading all those books. I also don't see any way to stop the phenomenon of the author getting his friends, family, and grad students to write good amazon reviews of his book.

Because of all this, I'm suspicious of any statistical method of analyzing user-submitted reviews. You just have no way of knowing which reviewers are honest. You really have to look at the individual reviews and see if what they say makes sense. Ebay feedback is an example of how silly this can all get, even in a community where people really do have long-term online identities that they have an interest in maintaining good reputations for. What the heck does it tell you if the seller has 99% positive feedback? Absolutely nothing. You have to read the 1% negative reviews and try to evaluate whether they sound reasonable.

A reflection of the times.

This is a serious reflection of our current times, where people's eyes gloss over if the concept at hand is not condensed into a convenient sound-byte.

I suppose you could call it the bleeding edge where complacency meets the loss of freedom and the fall of darkness where critical thought once stood.

Now there is enough probable demand to launch a startup designed to remove what minimal labor people are interested in dedicating to the quality of even their leisure time.

I'm sure many fantasize about strangling people this lazy/complacent, but honestly if they're unconscious enough not to care about their own toys, do they really possess a "life" for you to take from them?

I'm to lazy to read TFA

can someone boil it down to a couple sentences for me?

Re:I'm too lazy to read the summary

Could you boil it down to a word?

Forty-two

It's not summarization.

It's heavily templatized generation of language based on the automatically extracted sentiment data. The important difference here is that the language of the summary does not include phrases from the original user reviews. While this is a new twist on the old problem, automatic extraction of evaluation criteria and sentiment analysis in product reviews are not new. Heck, even Microsoft has a working system for that (electronics only):

http://search.live.com/products/?q=nuvi%20350%20GPS%20-%20Asian%20American%20(City%2FVehicle%2C%203.5%22%20LCD)&p1=%5BCommerceService+scenario%3D%22reviews%22+docid%3D%222BECBBF6F17C98618C2E%22+p%3D%2220df8fe62a9b4e9490993ff7b91032af%22%5D&wf=Commerce&FORM=ENCA [live.com]

See the bars on the left, and be sure to click through to the individual sentences. It's spooky how accurate that thing seems to be.

The problem with all these systems is that they're heavily domain dependent. You will use different language to write a review of a book than for kitchen appliance. In fact, you may even use different language from different kinds of books or different kinds of kitchen appliances. Worse yet, some things are notoriously difficult to accurately measure sentiment on. Once innuendo and sarcasm become frequent, all hope is lost - you need strong AI to figure that out.

This is not to say these systems are useless - to the contrary, they are very useful in their respective domains. This is just to say that the only new thing I see here is the generated blurb.

Re:

I think that Microsoft live product already has strong AI; it knows how to brown-nose so Microsoft doesn't kill it:

"Windows? Millennium Edition, or Windows Me, is the home operating system for PC?s that brings the richness and convenience of the digital world to your home. Windows Me is designed specifically for the home PC user. It represents the first major milestone towards advancing the vision of the Windows Division and further simplifying the computing experience for consumers. Windows Me delivers in

Mac OS X Summary Service

Built into Mac OS X is a Summarize Service (click on the application's menu and point to "Services" and choose Summarize) that is pretty cool. Unfortunately, Firefox was never coded to work with that OS X native service, so I had to copy and paste the text of that Ars Technica article into Text Edit and then use that service. But here is the resulting summary (I had to do as short a summary as possible):

While summarizing other types of non-review content isn't on the company's roadmap for now, Chakrabarti said the company is listening to user feedback. He also admitted that, "in many ways, summarizing facts is actually a lot easier than summarizing opinions." Pluribo's current focus on user-generated reviews and opinions is certainly a useful one, but I can see all kinds of other consumer applications for things like long Wikipedia entries and even articles like the one you're reading.

Ugh. The opposite of what I want.

Ugh.

This sounds like the opposite of how to get useful information out of reviews, and more like the "consumer products" equivalent to the automatic resume scanner.

You know the resume scanners I'm talking about -- the ones that circular-file the candidate who took three years off work to get his Ph.D. in cognitive science (and whose thesis is a perfect fit for your business plan), preferring, instead, the guy who listed "20 years PROLOG, PL/1, BASIC, C, 10 years C++, 5 years Java, MCSE, A+", because obviously the second guy triggers more buzzwords. Because the HR drone won't understand any of the resumes, he/she just picks whichever one the scanner selects, and that's typically the one with the fewest career gaps and the most buzzwords. ("But that other Ph.D guy only has one or two languages, this guy has six! And that Ph.D guy's been out of work for three years, so obviously nobody would hire him!")

Ten reviews reading "Works. Fast, cheap, lightweight" and three reviews reading "Doesn't work" don't tell me anything, other than that the product might have reliability issues.

One review reading "Didn't work the first time. The manual doesn't mention that you have to make sure the jumper is in the correct position first, and then it works. I own an XYZ-123 and this new product was at least as fast, but at about half the price. Weighs about a pound." tells me everything I need to know -- that the three people who claimed it didn't work almost certainly didn't know how to configure it correctly, and that the first seven reviewers never had a problem because they weren't part of the edge case.

Keywords in Context

I'm concerned how well the applet properly discerns the meaning of words in context. For example, just because I mention that a product is 'a portable laptop' does not mean I am impressed with it's size or weight... it's just the category the product falls in. But judging from the screenshots in the article, this exact error was committed by the plugin.

Reading natural language is hard, and I'm of the opinion that a Firefox plugin just won't cut it for understanding the nuanced opinions given by reviewers.

MacOS X: How to Summarize the Contents of Document

This document explains how to use the Summarize services available in Mac OS X applications.

If you have a long document, you can use the Summarize service to get a summary of the contents. For example, use this to get a short version of a long page on a Web site.

To get a summary of a document, select the text and choose Services from the application's menu, then choose Summarize.

If the application you are using doesn't support services, copy the text to a TextEdit document to get a summary.

Note: The information is this document comes from Mac OS X Help, the help system included with your computer. It is based on Mac OS X 10.1.2. If a different version is installed on your computer, choose Mac Help from the Help menu. Updated and expanded information may also be available in other Knowledge Base documents.

http://docs.info.apple.com/article.html?artnum=61336 [apple.com]

(but i know this feature was in OS 9 or earlier)

Re:How about a comment synopsis generator

I'd love to get a one page summary of all the informative, insightful and interesting comments.

[url=http://slashdot.org/~Rob+Kaper]Here you go[/url].

Re:How about a comment synopsis generator

Joke's on me, this time, I guess. *sigh*

Re:How about a comment synopsis generator

Man, you guys really don't know about Alterslash [alterslash.org] yet?