Why You Shouldn't Trust Internet Comments 180
sciencehabit writes "A new study suggests that all the reviews you read on Yelp and Amazon are easily manipulated. It's not that companies are stacking the deck, necessarily, it's that a few positive comments early on can influence future commenters. In fact, when researchers gamed the system on a real news aggregation site, the items received fake positive votes from the researchers were 32% more likely to receive more positive votes compared with a control (abstract). And those comments were no more likely than the control to be down-voted by the next viewer to see them. By the end of the study, positively manipulated comments got an overall boost of about 25%. However, the same did not hold true for negative manipulation. The ratings of comments that got a fake down vote were usually negated by an up vote by the next user to see them."
Survivor bias (Score:3, Insightful)
Nothing new here, move on...
Great article! (Score:2, Insightful)
This is by far the most insightful treatment I've ever read on this important issue. Everyone who does business on the Internet must read this valuable study.
Well sure (Score:5, Insightful)
My hypothesis:
1) Products with positive comments are more likely to be purchased.
2) People identify themselves by their choices, and no one wants to make a bad choice. Ergo, almost by definition, any choice people make is "the right one". At the very least, people are predisposed to liking what they spent money on.
Remember; When discussing all things retail, it's not how good the product is, but how well it satisfied the need. At least half the time, that need is largely imaginary.
Sure, I could RTFA, but this is more fun.
ethics problems (Score:4, Insightful)
Re:I don't think so... (Score:5, Insightful)
I'd say it's far more complicated then that.
If you have a bad experience and go to the product review and the other reviews are bad, you are apt to write a review confirming what you are reading.
On the other hand if you have a bad experience and all the other product reviews are good you may have a moment of self doubt (did I mess up with the product) which makes you less willing to post a negative review.
Perception vs actual rating (Score:5, Insightful)
When I check reviews, one of the first things I do is check the negative reviews. Why? Because half the one-stars are often jackasses with no clue what product and/or service they were buying. Other times, knowledgable and otherwise reasonable people have found the service or product being rated to be inadequate in some significant way.
And then I look for high ratings to see if they are reviewing the product in a reasonable manner. From there, I make my own decisions regarding the validity of both sides.
Anyone who decides just based on the stars/review-based numbers is a fool.
Re:Tell me about it (Score:4, Insightful)
Re:I don't think so... (Score:4, Insightful)
I hate to say this, but I find I am more likely to take the time to write a bad review than a good one. (Anger is a great motivator.) I assume others are like this as well so I read the negative reviews in that light.
Also, any review in all caps, good or bad, I automatically discard.
You all joke... (Score:5, Insightful)
Re:I don't think so... (Score:4, Insightful)
I hate to say this, but I find I am more likely to take the time to write a bad review than a good one. (Anger is a great motivator.) I assume others are like this as well so I read the negative reviews in that light.
Also, any review in all caps, good or bad, I automatically discard.
That's been my experience, too. Anger motivates you to want to do something, so people lash out on the comment board. People who are satisfied, by definition, aren't really motivated to take any additional steps.
Re:I only read negative comments (Score:5, Insightful)
Re:Not surprising (Score:5, Insightful)
Or maybe they just weren't visible to the moderators until a few upvotes brought them above the viewing threshold. Once visible, comments that happen to be genuinely insightful, informative etc, would get more upvotes because they deserve it. You don't have to invoke bandwagon effects to describe what you observe. Also, when I have moderator points, I tend to upvote good comments that *don't* have a high score because they are worth drawing attention to. Wasting moderator points on a "me too" upvote of a +5 comment is a poor use of the privilege. In my view, the purpose of moderation isn't to "skew" the discussion to reinforce the echo chamber., Rather moderation should improve the overall signal/noise ratio so threshold settings are actually meaningful.
Re:Survivor bias (Score:5, Insightful)
This is very true....and that is before you even get to the other question... even if they are not lieing, how do you know they are really a good commentor? Now, at this point, as we have had several blender's burn out, my wife is familiar with several of the models on the market. I honestly doubt there are too many other devices that we are as familiar with across different brands/models.
So what if you think this is the greatest cordless drill ever and the battery just goes forever. How do I know you are not basing that on a comparison with some cheap crap drill you bought in the mid 90s with a battery that shit the bed after a handful of recharges?
Not only that but, its rare that someone goes back and re-evalutes the product later. I have seen it...I have totally seen amazon reviews that said one thing, then had an edit explaining "Now that I have been using it for 6 months some issues have cropped up..."
Even less likely is that they get the product and use it before they up or downvote the comment. So really the only question on comment ratings is not really about accuracy but about whether it helped you decide to buy or not.
I generally look for posts of substance, that seem to actually have knowledge of what a product does and how it should work... but that are not so in depth as to look professionally writen, or by commenters that comment on 6 items a day (seriously, you really think I think you could possibly buy and evaluate that much crap?)
Also I find looking for the mid range star ratings is best. I generally skip right over the 5 star ratings for some 3 and 4 stars first, to get a feel for whats wrong with it. Often the features are less important than the defects, and its more about picking which defects I can live with than which features are best.
Look at the negative reviews (Score:4, Insightful)
Everytime I look to buy something I am not not an expert on or are on the fence about quality, I look up reviews and sort by rating. And I look at negative ratings first because a vendor or retailer won't pad a product with negative reviews. And even though there will always be negative reviews from people who dont have a clue or give little to no info, there are som rea gems out there that give you a clear picture of what you are getting into.
Obligatory anecdotes:
When looking on newegg I sort by lowest score first and read the reviews. You always have some dummies who obviously have no idea what they are doing and rate 1 star because of a mistake they made. But you also run across some genuinely informative negative reviews which are more influential to me than positive reviews. For example, I was looking to buy a uATX board from ECS that had the AMD bobcat CPU onboard. It was perfect, had extra PCI shots for SATA cards for a low power Linux server box. It turns out in a few of the negative reviews there was an IRQ bug that severely impacted performance. I was close to buying it but then scratched it off my list.
Recently I was also looking to purchase a generator from a coworker who bought it after hurricane sandy but never used it, the box is unopened. The brand name was Generac and I have owned two other Generac products, a power washer and a 4kw generator. Both of those machines went south after little use, the generators exhaust valve stuck open when the valve seal went bad and allowed oil to seep down the valve and seize it (I fixed that but it never ran quite right, stalled and was a bitch to start). The power washer engine needed its carb rebuilt and then the water pump blew a shaft seal. But that was 6+ years ago and I figured Generac got their shit together by now. After reading negative reviews on Amazon I came to understand that Generac will try to weasel out of warranty repairs and "authorized" repair shops frequently change as they get shafted after Generac refuses to reimburse them for warranty repairs already performed. There were also negative reviews that warned of blown stator coils after a few hours use and lemons that wouldn't start out of the box. Since the generator could not be returned to the original vendor I took a pass even though I would get a sweet deal ($200 off retail as he wanted to dump it). Maybe it would have worked fine but I didn't want to risk losing 800 bucks and damage my business reputation.
Re:Survivor bias (Score:4, Insightful)
I am the same way.
I'm also far more likely to review something if I disagree with the collective opinion on the site. I figure the 47th glowing review is useless but the 3rd bad one adds clarity.