Forgot your password?
typodupeerror
The Internet Communications

Why You Shouldn't Trust Internet Comments 180

Posted by Soulskill
from the because-they're-internet-comments dept.
sciencehabit writes "A new study suggests that all the reviews you read on Yelp and Amazon are easily manipulated. It's not that companies are stacking the deck, necessarily, it's that a few positive comments early on can influence future commenters. In fact, when researchers gamed the system on a real news aggregation site, the items received fake positive votes from the researchers were 32% more likely to receive more positive votes compared with a control (abstract). And those comments were no more likely than the control to be down-voted by the next viewer to see them. By the end of the study, positively manipulated comments got an overall boost of about 25%. However, the same did not hold true for negative manipulation. The ratings of comments that got a fake down vote were usually negated by an up vote by the next user to see them."
This discussion has been archived. No new comments can be posted.

Why You Shouldn't Trust Internet Comments

Comments Filter:
  • Excuse me?! (Score:5, Funny)

    by Seumas (6865) on Friday August 09, 2013 @10:56AM (#44520355)

    This comment is IMMENSELY trustworthy!

  • Survivor bias (Score:3, Insightful)

    by Anonymous Coward on Friday August 09, 2013 @10:57AM (#44520393)

    Nothing new here, move on...

    • by Z00L00K (682162)

      After a while you learn which comments to trust and which comments that are there to troll or spam.

      Of course - subtle trollings are harder to detect, but they may still contain a grain of truth too.

      • A subtle astroturfing is one thing, but many of the fake postings are pretty hilariously transparent. Good help is hard to find.

      • by Darinbob (1142669)

        I distrust them all. Honestly, I am baffled why so many people go to Yelp when it should be obvious to everyone how useless it is. No one posts if they get the appropriate level of service; so posts tend to come from extremes instead. If service is not up to part the negative reviews come out in force, because no one wastes their time online just to say "meh" the exaggerate how bad it was. And posts from the local A&W hamburger chain describing it in superlative terms can be discounted similarly. Y

    • by skids (119237)

      Actually there is something new: in addition to trolls, shills, and gadflies, some of the comments on the internet are placed there by people experimenting on internet comments.

  • Amazing (Score:5, Funny)

    by Anonymous Coward on Friday August 09, 2013 @10:58AM (#44520405)

    What an excellent report. I read this to my Son and he loved it, would recommend to anyone and definitely read again. 5*

  • by finkployd (12902) on Friday August 09, 2013 @10:58AM (#44520411) Homepage

    I've been here for over a decade and I STILL have yet to see Natalie Portman naked and petrified, despite all the hype.

    Also, I miss OOG The Caveman.

  • by neminem (561346) <neminemNO@SPAMgmail.com> on Friday August 09, 2013 @10:58AM (#44520413) Homepage

    is the best slashdot thread we've seen all week! I love it!

  • Great article! (Score:2, Insightful)

    by Anonymous Coward

    This is by far the most insightful treatment I've ever read on this important issue. Everyone who does business on the Internet must read this valuable study.

  • If someone has a bad experience, they will be pissed off and want to leave a bad comment. Often, that is the only recourse for a shitty service offered, or if a refund/exchange is no offered.

    If someone has a genuinely good experience, they may be happy and want the product/business to do well, so they will leave a comment to aid in that.

    I would think the only influence comes from choosing based on comments, not on leaving comments.

    • by PlusFiveTroll (754249) on Friday August 09, 2013 @11:10AM (#44520583) Homepage

      I'd say it's far more complicated then that.

      If you have a bad experience and go to the product review and the other reviews are bad, you are apt to write a review confirming what you are reading.

      On the other hand if you have a bad experience and all the other product reviews are good you may have a moment of self doubt (did I mess up with the product) which makes you less willing to post a negative review.

    • by JackieBrown (987087) <dbroome@gmail.com> on Friday August 09, 2013 @11:20AM (#44520713)

      I hate to say this, but I find I am more likely to take the time to write a bad review than a good one. (Anger is a great motivator.) I assume others are like this as well so I read the negative reviews in that light.

      Also, any review in all caps, good or bad, I automatically discard.

      • by Zordak (123132) on Friday August 09, 2013 @11:31AM (#44520891) Homepage Journal

        I hate to say this, but I find I am more likely to take the time to write a bad review than a good one. (Anger is a great motivator.) I assume others are like this as well so I read the negative reviews in that light.

        Also, any review in all caps, good or bad, I automatically discard.

        That's been my experience, too. Anger motivates you to want to do something, so people lash out on the comment board. People who are satisfied, by definition, aren't really motivated to take any additional steps.

  • Manipulation (Score:5, Informative)

    by Sockatume (732728) on Friday August 09, 2013 @10:59AM (#44520429)

    What's interesting about the study is:

    1) The manipulation was a single positive or negative vote applied at random immediately when the article went live.
    2) People would tend to correct false negatives, but amplify false positives.

    • by Noughmad (1044096)

      In other words, people tend to post review where there already are reviews. It's as though we were social animals.

    • Re:Manipulation (Score:5, Informative)

      by Sockatume (732728) on Friday August 09, 2013 @11:14AM (#44520647)

      My mistake, it's not even articles, it's just Reddit comments.

    • The issue is partially do the emotional attachment we have towards our stuff.

      We have a Mac We love our Mac, We have Linux we love Linux....
      We like reinforcement by others that we had made the right purchasing decision.
      The stuff we tend to hate is the stuff we didn't have a decision or a personal investment in (Such as your work Computer)

      It is very rare that we hate the products that we own, unless it is time to replace them, or we bought a lemon.
      Otherwise we have an emotional attachment to them and will ra

  • by captaindomon (870655) on Friday August 09, 2013 @11:02AM (#44520473)
    I trust the article that there may be subtle changes in future comments due to past comments. However, there is still a very valid difference between a 5-star item with 2,000 comments and a 1-star item with three or four comments, and that is good enough for me.
    • by Sockatume (732728)

      It's not even comments. The paper refers to "upvotes" which makes it pretty clear that the study took place on Reddit and involved incrementing/decrementing the score when an article went live, by a single point.

  • Well sure (Score:5, Insightful)

    by grasshoppa (657393) <skennedy@nOspaM.tpno-co.org> on Friday August 09, 2013 @11:05AM (#44520509) Homepage

    My hypothesis:

    1) Products with positive comments are more likely to be purchased.
    2) People identify themselves by their choices, and no one wants to make a bad choice. Ergo, almost by definition, any choice people make is "the right one". At the very least, people are predisposed to liking what they spent money on.

    Remember; When discussing all things retail, it's not how good the product is, but how well it satisfied the need. At least half the time, that need is largely imaginary.

    Sure, I could RTFA, but this is more fun.

    • by c0d3g33k (102699)

      I agree with point 1, though it is really just stating the obvious, so isn't saying much.

      I'd have to disagree with point 2. It could, perhaps, apply to products whose enjoyment is primarily a matter of taste or the response they elicit from other people, such as fashionable clothing. I can imagine situations where a poorly researched impulse purchase of an expensive item like an automobile could lead to self-deluding rationalization to justify the purchase. I'll grant you that. For more functional items

      • I'd suggest it's a behavior that's related to cost. The more something cost, perhaps even the more of a luxury item it is, the more likely someone is to be biased.

    • by 6ULDV8 (226100) on Friday August 09, 2013 @03:31PM (#44524195)

      people are predisposed to liking what they spent money on.

      I don't know about that. I spent a bunch of money on an ex-wife and I don't like her much at all.

  • ethics problems (Score:4, Insightful)

    by doom (14564) <doom@kzsu.stanford.edu> on Friday August 09, 2013 @11:08AM (#44520555) Homepage Journal
    I'm always glad to hear about research like this myself, but this has severe ethics problems. You don't con people to show how easy it is to con the people. I know that rationalization is popular with some segment of you "hackers" out there, but whenever social scientists do this, they end up getting hasled about it.
    • by timeOday (582209)
      It doesn't really matter, because markets are always lightyears ahead of what researchers are able to prove. Rhetoric has ALWAYS been about making what you like seem "normal", to be the default. You second-guess yourself when you're an outlier. This is exactly what hype (or buzz) is all about. It's why advertisers say "X million people can't be wrong!" Or, watch any politician and they almost always say "the American people want (whatever it is I espouse)." Studying these things formally just allows t
  • by intermodal (534361) on Friday August 09, 2013 @11:10AM (#44520591) Homepage Journal

    When I check reviews, one of the first things I do is check the negative reviews. Why? Because half the one-stars are often jackasses with no clue what product and/or service they were buying. Other times, knowledgable and otherwise reasonable people have found the service or product being rated to be inadequate in some significant way.

    And then I look for high ratings to see if they are reviewing the product in a reasonable manner. From there, I make my own decisions regarding the validity of both sides.

    Anyone who decides just based on the stars/review-based numbers is a fool.

    • I'll admit I'm tempted to look at overall number of stars, and assume a 4 star place is better than a 2 star place. But I usually end up looking more closely (because ALL the restaurants in an area will be suspiciously highly rated) at the negative reviews. Like you, I try to judge the relevance of the complaint. For example, if the worst thing that anyone has to say about a restaurant is that service is a little slow on Saturday nights and that they had trouble seating your party of 10 without a reservatio
    • Agreed - In similar vein, I have seen one-star reviews of restaurants stating that 'the line/wait was too long", meaning they never even *tried* the place;
      • Agreed - In similar vein, I have seen one-star reviews of restaurants stating that 'the line/wait was too long", meaning they never even *tried* the place;

        I am reminded of the Yogi Berra quote: "Nobody goes there anymore. It's too crowded."

    • by Deflagro (187160)

      I do that exact same thing. Most of the "HUGE ISSUES ZOMG" are either some preference that didn't meet personal expectations or just plain 'ol PEBKAC. I also take note of the volume of reviews too. If 5000 people rank a product 4 stars, that's a lot of use cases to check out. If a bunch of people report the same failure, it points out commonality and such.

      I am kind of OCD about making important purchases though :P

    • My favorite thing to do is read all the lengthy reviews. Someone who goes in depth into the product can give valuable feedback. Also, when someone says they've had it for a few months or something (rather than "I just got it 5 minutes ago and it's SO FUN and hasn't broken! Exceptional quality!") and are reviewing it after using it regularly ... that sort of thing. In other words, reviewing actual usage rather than reviewing how well it was shipped or packaged or how it "feels" when they first opened it
    • I once saw a review where the buyer gave it one star because the item he ordered wasn't really what he wanted.

      Yes, he ordered an item, they delivered it promptly, but he decided he really should have ordered something else and so he gave it one star. You just can't win with some people.

      • by Roogna (9643)

        I wrote a iOS game that was out for a few years. We got a 1 star review because we "weren't Tetris". Except the game wasn't Tetris, didn't claim to be Tetris, and had absolutely nothing to do with Tetris except for being vaguely in a similar genre of "real time puzzle game".

  • Probably one of the best stories EVAR posted in the history of Slashdot, superb! 5/5 stars! My friends and I all read the story at work, then we ordered Chinese Food. The food arrived SOOPER early and was delicious. The Orange Beef and Sweet and Sour Chicken were DELICIOUS! I should know, I own the restaurant I'm posting this for on Yelp!
  • Pressured by vendors (Score:5, Interesting)

    by stevegee58 (1179505) on Friday August 09, 2013 @11:16AM (#44520673) Journal
    My wife has posted negative reviews on a certain vendor's website (from real product experience, not trolling) when a product she bought was unsatisfactory.
    This elicited an immediate email from customer service offering various deals to bribe/entice her to change or withdraw the review. Companies are free to do what they wish on their website but that still struck me as disingenuous.
    • Hey, at least your wife didnt buy a GoPro [petapixel.com] and try to review it.
    • by neminem (561346) <neminemNO@SPAMgmail.com> on Friday August 09, 2013 @11:30AM (#44520863) Homepage

      I actually have no problem with that - if done right. If the response was "I'll give you free stuff, but only if you remove your review", then yes, that is super sketchy. But a lot of times it's more just "I'm sorry that happened. Would you like to give us another shot on us? It was probably a fluke." And that is exactly what customer service *should* be like. If you go back and it was a fluke, then you change your review, and everyone's happy. If you go back and it happens again, then that company clearly needs to pay their other departments as much as their customer service department, but I'd still rather that than a response of "tough luck, go away."

    • by mwvdlee (775178) on Friday August 09, 2013 @11:46AM (#44521107) Homepage

      This is the way companies work these days.

      Have a problem with a product? Don't navigate through dozens of useless pages on a support site, don't wait for an hour trying to get through to their helpdesk; post a complaint on twitter, wait a few minutes and they will contact you.

      I wish this was a joke, but this is honestly how I deal with some companies nowadays.

    • by bazorg (911295)

      This is why reviews are more valuable when managed by a third party that is integrated into the retail website, while keeping their own name and reputation on the line for being independent reviews "provider".

      Because of sheer volume of sales, Amazon reviews are an obvious starting point, but I normally have to exclude the 5 stars and 1 starts in order to understand if the product has serious problems or if it's the user that failed to RTFM. Going direct to the website of the reviews company is a very useful

  • You all joke... (Score:5, Insightful)

    by barlevg (2111272) on Friday August 09, 2013 @11:21AM (#44520731)
    But how much less likely are you to down-mod a score-5 tweet than a score-1? And how much more likely are you to read-and-upvote a red firehose submission than an indigo?
  • Isn't this *everyone's* rating on eBay, and yet eBay is 99% populated by crooks at this point?

  • by omnichad (1198475) on Friday August 09, 2013 @11:22AM (#44520759) Homepage

    Your hosts file comments are not trustworthy.

  • The subject of this comment is true.

  • by loufoque (1400831) on Friday August 09, 2013 @11:25AM (#44520795)

    There is no value in positive comments.
    If you're considering to buy something, you want to know where it fails, not where it succeeds.

    Moreover I personally would never leave a positive comment. If it works as advertised, life can go on as normal. If it doesn't, then I leave a comment.

    • by cusco (717999) <brian,bixby&gmail,com> on Friday August 09, 2013 @11:51AM (#44521183)
      I'll leave positive comments, normally when I receive outstanding customer service. People should be recognized when they do good work, and far too often management doesn't bother unless they have input from outside. I'll also ask to speak to their supervisor if the situation warrants it, and let them know if someone has done something outstanding.
    • by c0d3g33k (102699)

      There is no value in positive comments

      Of course there is. Thoughtful positive comments that describe the hands-on experience of using a product and how well the advertised features are implemented can be extremely useful. There are plenty of product categories where all the top products function well, but have different strengths/weaknesses/usage_patterns/quirks. Detailed positive reviews are quite valuable because they provide details not offered by the advertising/product description - these details help the purchaser choose the product be

      • by loufoque (1400831)

        We're talking about Amazon-type reviews here, no fully-fledged detailed reviews.

        • by Endo13 (1000782)

          Same still holds. If you were correct and everyone followed what you claim, every item on Amazon would be rated one star. Positive ratings and reviews are necessary to offset the inevitable negative ones, at the very least.

        • by c0d3g33k (102699)

          I didn't explicitly say so, but I was recollecting helpful Amazon reviews when I wrote my comment. They certainly vary in detail - most reviewers don't give a fully detailed review, but they provide something more than "I love it". Often they tell a story about whether a product solved the problem they bought it for (did the pet hair attachment for the vacuum actually work for them and their dog? Did the crockpot burn the food after running all day?). If there are more than a few reviews, different peop

  • by dkleinsc (563838) on Friday August 09, 2013 @11:25AM (#44520805) Homepage

    To use an example from /., it's not at all uncommon to see a comment go completely unnoticed from moderators for an hour, and then get a +1 from somebody, and within 30 minutes have gone from Score:1 or Score:2 to Score:5.

    Bandwagon effects are quite well-known. After all, all your friends are paying attention to them! It seems to be a useful psychological reaction: If all your friends and family are jumping off the bridge, chances are you will too on the theory that they probably have a good reason to do so.

    • Re:Not surprising (Score:5, Insightful)

      by c0d3g33k (102699) on Friday August 09, 2013 @12:13PM (#44521513)

      Or maybe they just weren't visible to the moderators until a few upvotes brought them above the viewing threshold. Once visible, comments that happen to be genuinely insightful, informative etc, would get more upvotes because they deserve it. You don't have to invoke bandwagon effects to describe what you observe. Also, when I have moderator points, I tend to upvote good comments that *don't* have a high score because they are worth drawing attention to. Wasting moderator points on a "me too" upvote of a +5 comment is a poor use of the privilege. In my view, the purpose of moderation isn't to "skew" the discussion to reinforce the echo chamber., Rather moderation should improve the overall signal/noise ratio so threshold settings are actually meaningful.

    • by Endo13 (1000782)

      Oblig: http://xkcd.com/1170/ [xkcd.com]

  • (Just checking its validity)
  • I won't say who, but a few years ago I had a temporary SQL admin contract with a very large company. I remember the day they hired a good 20+ people who's job it was to do nothing but post fake reviews and make fake blog posts all day long. The called it "Professional Blogging". I've been disgusted by the lack of ethics and misguided moral compass of companies before, but to actually see something like that go down really shocked and disgusted me to my core.
  • Assuming the researched posted fake reviews real businesses and products, isn't there an ethic issue with this research.

    I wouldn't like to be the owner of an otherwise good restaurant that ended up getting lots of bad reviews just because some researchers' coin flip decided my restaurant should get bad reviews for their research.

    It just seems a tad unethical to try and randomly ruin businesses for a research paper.

    Perhaps we should do some research on what happens when you randomly bankrupt researchers.

  • by raymorris (2726007) on Friday August 09, 2013 @12:13PM (#44521511)
    I've certainly noticed that when I am one of the first to post on a Slashdot story, it tends to set the tone for many comments to follow. This was perhaps most noticeable when I pointed out all the BS in a certain anti-patent propaganda story. Commenters did some critical thinking and mostly agreed the story was a load of BS. It's apparent from other similar stories that without someone setting the tone, Slashdot readers generally revel in anti-patent propaganda, expanding it beyond the already BS claims in TFA of the day.
  • I only trust Amazon product comments if they are vetted and posted to http://leasthelpful.com/ [leasthelpful.com]

  • A++++ wuld read again

    And probably will when it gets duped in three days time.

    I had to retract one + because of a missing word in the summary:

    In fact, when researchers gamed the system on a real news aggregation site, the items that received fake positive votes from the researchers were 32% more likely to receive more positive votes compared with a control

  • "Accentuate the positive, eliminate the negative" says the song. Same thing with upvotes.

  • This thread burned down my house, killed all my family, and kicked my dog. This is the worst thread ever, please don't read it or you can end up like me!
  • by LoRdTAW (99712) on Friday August 09, 2013 @01:35PM (#44522691)

    Everytime I look to buy something I am not not an expert on or are on the fence about quality, I look up reviews and sort by rating. And I look at negative ratings first because a vendor or retailer won't pad a product with negative reviews. And even though there will always be negative reviews from people who dont have a clue or give little to no info, there are som rea gems out there that give you a clear picture of what you are getting into.

    Obligatory anecdotes:
    When looking on newegg I sort by lowest score first and read the reviews. You always have some dummies who obviously have no idea what they are doing and rate 1 star because of a mistake they made. But you also run across some genuinely informative negative reviews which are more influential to me than positive reviews. For example, I was looking to buy a uATX board from ECS that had the AMD bobcat CPU onboard. It was perfect, had extra PCI shots for SATA cards for a low power Linux server box. It turns out in a few of the negative reviews there was an IRQ bug that severely impacted performance. I was close to buying it but then scratched it off my list.

    Recently I was also looking to purchase a generator from a coworker who bought it after hurricane sandy but never used it, the box is unopened. The brand name was Generac and I have owned two other Generac products, a power washer and a 4kw generator. Both of those machines went south after little use, the generators exhaust valve stuck open when the valve seal went bad and allowed oil to seep down the valve and seize it (I fixed that but it never ran quite right, stalled and was a bitch to start). The power washer engine needed its carb rebuilt and then the water pump blew a shaft seal. But that was 6+ years ago and I figured Generac got their shit together by now. After reading negative reviews on Amazon I came to understand that Generac will try to weasel out of warranty repairs and "authorized" repair shops frequently change as they get shafted after Generac refuses to reimburse them for warranty repairs already performed. There were also negative reviews that warned of blown stator coils after a few hours use and lemons that wouldn't start out of the box. Since the generator could not be returned to the original vendor I took a pass even though I would get a sweet deal ($200 off retail as he wanted to dump it). Maybe it would have worked fine but I didn't want to risk losing 800 bucks and damage my business reputation.

    • by 0111 1110 (518466)

      And I look at negative ratings first because a vendor or retailer won't pad a product with negative reviews.

      Right. Because retailers are too stupid to think of the idea of paying shills to leave negative reviews of their competitors. There are also fake negative reviews.

  • This post's subject is untrustworthy
  • You can see the phenomenon on Slashdot itself.

  • The article only talks about how reviews get rated after an initial rating, not about how subsequent reviews are good/bad if the initial review is good/bad. That would be the 'xxx people found this review helpful' - giving it a bunch of helpful ratings initially make more people likely to vote it helpful.

    On a slightly related note, I wish Amazon would allow viewers to only view rating histograms of 'Amazon verified purchasers'. I remember when Amazon hadn't even released the Fire (except to Vine members -

What this country needs is a dime that will buy a good five-cent bagel.

Working...