Follow Slashdot blog updates by subscribing to our blog RSS feed

 


Forgot your password?
Close
typodupeerror
×
Social Networks The Internet

Data Mining Shows How Down-Voting Leads To Vicious Circle of Negative Feedback 293

Posted by Soulskill from the vicious-circles-are-the-best-circles dept.
KentuckyFC writes: "In behavioral psychology, the theory of operant conditioning is the notion that an individual's future behavior is determined by the punishments and rewards he or she has received in the past. It means that specific patterns of behavior can be induced by punishing unwanted actions while rewarding others. While the theory is more than 80 years old, it is hard at work in the 21st century in the form of up- and down-votes — or likes and dislikes — on social networks. But does this form of reward and punishment actually deter unwanted actions while encouraging good behavior? Now a new study of the way voting influences online behavior has revealed the answer. The conclusion: negative feedback leads to behavioral changes that are hugely detrimental to the community. Not only do authors of negatively-evaluated content contribute more but their future posts are of lower quality and are perceived by the community as such. What's more, these authors are more likely to evaluate fellow users negatively in future, creating a vicious circle of negative feedback. By contrast, positive feedback does not influence authors much at all. That's exactly the opposite of what operant conditioning theory predicts. The researchers have a better suggestion for social networks: 'Given that users who receive no feedback post less frequently, a potentially effective strategy could be to ignore undesired behavior and provide no feedback at all.' Would Slashdotters agree?"
This discussion has been archived. No new comments can be posted.

Data Mining Shows How Down-Voting Leads To Vicious Circle of Negative Feedback More

Data Mining Shows How Down-Voting Leads To Vicious Circle of Negative Feedback

Comments Filter:

  • BS (Score:5, Funny)

    by marcello_dl ( 667940 ) on Saturday May 17, 2014 @03:56PM (#47027639) Homepage Journal

    This story sucks.
    Let the game begin :)

    • Re:BS (Score:5, Interesting)

      by the_povinator ( 936048 ) on Saturday May 17, 2014 @04:30PM (#47027917) Homepage
      I read TFA, and unfortunately the research is very weak. They did not do a proper randomized study, they merely trained a classifier to figure out how good they thought a post was, and then divided up posts into pairs where their classifier thought they were the same but the feedback was different. They assume that their classifier is accurate, i.e. really reflects the goodness of a post. The rest of their research follows from this assumption.

      If it had been a proper randomized study (i.e. roll a dice and up/down vote posts) I could have believed it.

    • Re: (Score:2)

      by sconeu ( 64226 )

      To achieve optimal results, I'm going to ignore this story.... Oh. Shit.

  • In other words... (Score:5, Insightful)

    by Dr Fro ( 169927 ) on Saturday May 17, 2014 @03:57PM (#47027647) Homepage

    Don't feed the trolls?

    • Don't feed the trolls?

      I'd agree with not engaging them. At least not the trolls we have today.

      But mod'ing them down? I like that. It means I don't have to wade through hundreds of trash messages to find anything worth reading.

      And a clarification. "Troll" is NOT the same as "I have a different opinion".

      • If you mod a troll post down to -100 or more like on reddit, that means 100 people read the post and that is a huge boost to trolls that want attention.

        On the other hand, if you have no means to downvote like the majority of hacker news, then it rewards cliches that upvote their own content even though it sucks and you get stupid "Hello World... in Go" posts every day.

      • Disagree when you have a published study supporting your side.

        If it actually hurts the community, you're going to have to get over it. A troll voted up definitely needs to be flagged as troll, either with a reply or moderation, but otherwise behavioral science is pretty much telling you to shut it.

        The end result is, in my experience with 4 or 5 user names here since 2001, is "you aren't listening to me, it won't matter, I'll shit on your floor" acting out.

        Remember the "fuck beta" stuff where people who got

      • Re:I have to disagree with TFA. (Score:4, Interesting)

        by swillden ( 191260 ) <shawn-ds@willden.org> on Saturday May 17, 2014 @05:35PM (#47028249) Journal

        But mod'ing them down? I like that. It means I don't have to wade through hundreds of trash messages to find anything worth reading.

        On slashdot, I think negative feedback does result in more trollish activity, but it also pushes the activity below the threshold at which most people read, so the community doesn't see it and isn't damaged by it. Trolls also don't get mod points so they can't visit their wrath on others.

        All in all, I think it works pretty well. I'll leave it to others to discuss if the mechanism to suppress trolls has negative side effects.

    • Re: (Score:2)

      by AK Marc ( 707885 )
      No, the replies were not studied, only moderation. +1 has no effect on the poster's future comments, but -1 makes them more troll-like. The "solution" seems to be only allow +1, and not downvotes (or mask downvotes past 0). That will result in the best evaluation of comments, while not encouraging bad behavior.
      • i agree, the up vote / down vote system works well here because people who have low karma can't post as much as people with good karma. another solution is the "alternate reality" approach, where a troll's posts are visible to him but not to others. he thinks he's being ignored.

      • (or mask downvotes past 0)

        So, kind of like Slashdot does, where the worst rating is -1, but the best is +5?

        You might be right. But that system is only helpful if -1 posts can be filtered out of my feed (which Slashdot allows).

        I'm just now reading this study, and I'm not sure it's really that well designed. I also wonder if with the proliferation of commercial speech in social media, if there's not a strong desire on the part of corporations to eliminate negative moderation of their speech. It wouldn't b

        • Re: (Score:2)

          by AK Marc ( 707885 )
          Perhaps the fix is more slashdot-like. mod down for reason. -1 commercial advertisement. Then let me set commercial advertisement to -3. I'll almost never see it, if I don't want to see astroturfing.

      • Other suggestions (Score:5, Interesting)

        by justthinkit ( 954982 ) <floyd@just-think-it.com> on Saturday May 17, 2014 @06:14PM (#47028441) Homepage Journal
        6 years ago I suggested that any negative mod should cost you 3 mod points [slashdot.org].

        2 years ago I said On political threads, all comments should have the same rating. [slashdot.org]

        Today I would add that maybe /. should increase the maximum points a post can accumulate. Giving more "upside" to the discussion.

  • meta. (Score:4, Interesting)

    by stoploss ( 2842505 ) on Saturday May 17, 2014 @03:58PM (#47027657)

    I look forward to observing the many ironic and humorous mods this topic will induce. In fact, the act of moderation itself may be the actual discussion more so than any of the content.

    I would mod my own post as insightful troll, for example. I mean, this is just pandering, right?

    • Thanks for all the mods, everyone! The moderation log message from the Slashdot backend system report was hilarious to read.

      I wish the complete mod log were visible on the comment, in chronological order (Funny displaced by Flamebait displaced by Insightful, Troll, Interesting, and so it iterated). It really was a very amusing meta conversation.

  • No shit, Sherlock.. (Score:3)

    by Xiph1980 ( 944189 ) on Saturday May 17, 2014 @04:00PM (#47027677)
    In the category of "No shit, sherlock" research....

    Don't feed the trolls. Thought this was fairly common knowledge...

    • Re: (Score:3, Informative)

      by timeOday ( 582209 )
      On slashdot, downmodding trolls serves the purpose of filtering out the messages you they are mostly invisible if you don't want to see them. I call that a benefit.
    • the finding here is that "don't feed the trolls" can be rigorously demonstrated as true.

  • Common sense (Score:2, Interesting)

    by Anonymous Coward

    The findings seem to be common sense. Or, as the saying goes, "Do not feed the trolls". Alternatively, the popular wisdom is, "Ignore them and they will go away." I have seen this in action on many forums. Debating a troll or a bad writer will just cause them to post more and more, they become more combative. Ignoring a troll or someone who is behaving badly and they usually pack up and go someone else to annoy other people. Postive feedback can encourage additional posting, at least that has been my experi

    • Re: (Score:3, Interesting)

      by Baby Duck ( 176251 )

      Don't Feed the Trolls means don't engage them will novel comments you crafted yourself. The impersonal act of downvoting without comment doesn't fit in their diet.

      The researchers are really bad at establishing causation. People who generate content awful enough that others actually bother to make the clicks to Downvote ... are more likely to make inferior content again in the future. It's because they suck at critical thinking and/or writing. No combination or up or down voting will magically bestow these s

    • They don't always go away. One troll amazingly sat on the rec.bicycles.* groups for years and hastened their decline with his ceaseless vitriol.

  • Let's hope it can be replicated.

  • The behavior described in the study is expected. We've all seen the effects of giving trolls the attention they crave.

    A more interesting study would be into the coupled nature of troller and trollee. Why are some incapable of ignoring negative provocateurs? We are told "Do not feed the trolls." But some cannot resist. Why are some incapable of letting the troll starve and vanish?

    • Re:Do not feed the trolls. (Score:4, Insightful)

      by BergZ ( 1680594 ) on Saturday May 17, 2014 @07:20PM (#47028735)

      We are told "Do not feed the trolls." But some cannot resist. Why are some incapable of letting the troll starve and vanish?

      ... because we are also told "the cure for bad speech is more speech".
      When a troll posts misinformation (especially those long debunked arguments) I think the people who reply are not attempting to convince the troll (trolls can't be convinced):
      They're trying to persuade the reasonable readers with facts and better information.

  • On Usenet there is the killfile, so at least people who know what they're doing can trash the crap, and these would typically be the kind of people the negatively-rated posters would have been trying to impress. The problem remains that newcomers and those unaware of the 'k' button remain exposed to the idiocies of such posters.

    Elsewhere, it's not clear whom the negatively-rated posters are trying to impress — if anyone. More likely they're just trying to get something, anything, out on the interwebs

    • it's not clear whom the negatively-rated posters are trying to impress

      They aren't trying to impress anybody. Since nobody knows before they post whether any given post is going to be upvoted or downvoted (OK, it is possible: simple even to craft posts that will reliably achieve broad acceptance or anger on pretty much any forum), there's little incentive for trying to impress. It's also impossible to predict which forum members will see a post and which will choose to judge it by voting.

      As it is, I suspect that a large number of up or down votes are obtained simply from th

  • Slashdot's moderating system (Score:5, Insightful)

    by Solandri ( 704621 ) on Saturday May 17, 2014 @04:22PM (#47027843)
    That's the criticism, well maybe that's too strong a word... That's the critique I've had of Slashdot's moderating system. By allowing both up-votes and down-votes, you create a system where the voice of the majority can drown out the voice of the minority. I've often seen people here express the mistaken belief that if a minority viewpoint is introspective or informative, it will survive the unfair downvotes and rise to the top. It doesn't work that way.

    The average ranking is not rank = up - down. It's rank = p1*up - p2*down. Where p1 is the size of the population which would rank it up, and p2 is the size of the population which would rank it down. A minority viewpoint consequently gets a disproportionate number of unfair downvotes simply because it's a minority viewpoint, and thus has to garner a lot more upvotes just to obtain an equal ranking to a majority viewpoint.

    For an apolitical, non-religious example, consider Windows vs. Linux. Say Windows users outnumber Linux users 50:1. Now imagine if a search engine let you rate search results based on whether they were useful or not useful, which is then used to prioritize subsequent search results. In every population, there's going to be an idiot segment who votes stuff down simply because they don't like it, not because it was inaccurate or irrelevant it was to their query. Consequently, if a search for hard disk repartitioning brings up four Windows sites and one Linux site as the top results, the Linux site is going to have 50x as many downvotes from those idiot users who never specified Windows in their search but were upset that an "irrelevant" Linux site was included in the search results. If the idiot segment of the Windows population exceeds 2% (numerically equivalent to 100% of the Linux population), that Linux site will end up with a negative rating regardless of how useful or informative it is.

    I say "criticism" is too strong a word because neither way is the "right" way to do it. They are just different. A moderating/ranking system which only allows upvotes simply generates different results from a moderating system which allows both upvotes and downvotes. Sometimes the former is more useful; sometimes the latter is more useful. The important thing is to understand the limitations of both and how it will bias the rankings, and not fall into the mistaken belief that a minority viewpoint has just as easy a time reaching +5 on Slashdot as a majority viewpoint. If a contrary viewpoint reaches +5 on Slashdot, it must be making a helluva good point.

    • Re: (Score:2)

      by AmiMoJo ( 196126 ) *

      For an apolitical, non-religious example, consider Windows vs. Linux.

      Surely that's the greatest and most politicised holy war ever? You make a good point though.

      Anyway, when I read TFS (no, of course I didn't RTFA) I immediately thought of Bennett. Somehow the story moderating system (firehose) fails and lets through a lot of his stories, which are always ripped to shreds by the comments. This only seems to encourage an even lower standard in the future.

      • Re: (Score:2)

        by fyngyrz ( 762201 )

        Somehow the story moderating system (firehose) fails

        Somehow? Are you not aware that firehose is advisory only, and that the rankings attained in it have no effect upon which stories post?

    • I disagree (Score:4, Interesting)

      by SuperKendall ( 25149 ) on Saturday May 17, 2014 @04:46PM (#47028011)

      Yes larger population could in theory tromp down a smaller one.

      But generally a larger population is more complacent and less likely to do anything, where a smaller population is more vigorous.

      I've voiced some unpopular opinions here. Yes sometimes I'm modded down. But pretty often I'm also modded up, so on average I feel the result is actually pretty fair - over time my voice is heard, despite blips of silence.

      Read at -1 for a bit before you truly claim that down-moderation is not needed... or at least if not down, some people just need an off switch.

      I think a combination of user moderation along with a handful of overseers that address the more egregious moderation abuses by the mobs, would be the way to go.

      • Re:I disagree (Score:5, Interesting)

        by dbc ( 135354 ) on Saturday May 17, 2014 @05:00PM (#47028059)

        What I find interesting is that over the years here on Slashdot, when I've posted an unpopular opinion it tends to simply get ignored. But.... unpopular *data*, now that is what brings out the pitchforks and torches. There is nothing that angers people so much as to be confronted with uncomfortable facts.

        • Re: (Score:2)

          by mrbcs ( 737902 )
          My favourite "opinion" that consistently gets flamed, down-modded etc:

          Nobody landed on the moon. Man that one sure pisses people off.

          Go from +2 to -1 troll in seconds.. :-)

        • Re:I disagree (Score:5, Interesting)

          by AthanasiusKircher ( 1333179 ) on Saturday May 17, 2014 @07:49PM (#47028907)

          What I find interesting is that over the years here on Slashdot, when I've posted an unpopular opinion it tends to simply get ignored. But.... unpopular *data*, now that is what brings out the pitchforks and torches. There is nothing that angers people so much as to be confronted with uncomfortable facts.

          Funny -- I've had the exact opposite experience. If I contradict a popular post on a controversial topic without evidence, it is ignored. If I cite reliable sources to backup my opinion, it often gets modded up.

          I have seen situations where people get downmodded or ignored for posting "facts" from unreliable sources, like conspiracy theories or some quack website. Or they only cite their own "data," which is often just speculation or anecdote.

          I'm not saying it doesn't happen -- but I've contradicted a LOT of posts around here that had already been modded up as "+5 insightful," because the parent was just making crap up, and I responded with a reasoned argument and links to back it up. Unless you're a jerk or your data is of the "tin-foil hat" variety, I've seen the behavior you cite quite rarely... at least in my experience.

      • I agree with the parent poster. I feel that my voice is rarely silenced due to simple unpopularity. Browsing at -1 indicates mostly that -1 posters suck. More common than this is that my posts are average (in the noise), which is probably an accurate reflection of my posting nature (small additions filling the the corners or highlighting a previous argument).

        On the moderation side, I rarely downmoderate. I downmod in one of a very few cases: poster is a jerk/troll, poster has contributed nothing, poster

    • I think you're onto something about up/down votes. Reddit has a system where you can sort by "controversial" but that in itself is a problem since it's just a pain in the butt to have to sort through two different systems of moderating.

      The one system I REALLY dislike is the only positive system of upvotes. The most obvious problem is there's little means to correct information that turns out to be innaccurate.

      Say someone posts something that initially looks extremely promissing and gets highly rated. Som

      • The one system I REALLY dislike is the only positive system of upvotes. The most obvious problem is there's little means to correct information that turns out to be innaccurate.

        The system Disqus recently adopted represents a reasonable compromise. While logged-in users can still upvote and downvote, only the upvotes are shown publicly. Downvotes still affect the placement of the comment on the page, but since you can't see whether your comment has been downvoted the negative feedback effect described in the article is probably averted.

        Another option might be to limit negative moderation to well-moderated replies. When replying to another comment, you could check a box to say that

    • Re: (Score:2)

      by wjcofkc ( 964165 )
      When making an insightful comment that represents a minority opinion bound to get modded down, always start your post with something along the lines of, "I know I am going to get modded down for this." or "Mod me down all you want, but..." I have been here for a very long time and recognize that as a genuinely working strategy. I picked up on it a very long time ago. Not only have I seen it work hundreds of times, I have used it myself. That is some psychology I would like explained.

      So mod me down all you
    • It would be interesting to add a feature that would allow filtering comments based on their number of mods overall. Both upmods and downmods would count the same. The more controversial comments would really stick out. A good compromise between troll filtering and making unpopular opinions visible might be at least a score of 3 OR at least 3 mods total.

    • Re: (Score:2)

      by Kjella ( 173770 )

      While you make many good points, the fact that the moderation is capped at -1 and +5 changes the dynamic considerably, you don't have the uncatchable +368 up-votes and irredeemable -78 down-votes. "Disputed" votes tend to get a voted down then ignored, then a few positive votes bring them back into view of the masses where get voted down again but this cycle keeps it close to the "surface" and many people still see the disputed arguments. Same with knocking a highly rated post down a notch, it really makes

    • There is one situation that doesn't follow your fairly elegant theory: when someone posts a minority view as an AC.

      To avoid mod-stalker trolls, I've resorted to this on a considerable number of occasions.

      What happens to the minority AC comments I've posted? 99% of the time absolutely nothing happens.

      To me this shows that mod stalkers are the problem on slashdot. They won't want to waste a mod point on an AC, as it has no long term consequence. Put your user ID to something ten times more tame, and

    • > The average ranking is not rank = up - down. It's rank = p1*up - p2*down. Where p1 is the size of the population which would rank it up, and p2 is the size of the population which would rank it down. A minority viewpoint consequently gets a disproportionate number of unfair downvotes simply because it's a minority viewpoint, and thus has to garner a lot more upvotes just to obtain an equal ranking to a majority viewpoint.

      You've just discussed / discovered the fundamental problem with a democracy. :-)

  • authors of negatively-evaluated content contribute more

    So if you want to increase the number of posts to your forum, down-vote everybody?

    I guess this is the problem when people try to apply the psychology of the real-world to entirely made-up worlds, or forums. Places where nobody really has any idea about the true identity (or identities) of the participants - and where reputation counts for little: since anyone can "press the button" and start again with a new identity - placing a value of zero on their forum-persona's reputation.

    • You cannot really be active in a forum and start over as too many people will find similarities in your posting styles and wording and connect the dots. You can usually find out who has sock puppet accounts in much the same way, people will make mistakes and allow the account's styles to bleed into each other giving someone notice if they interact a lot with them.

      That being said, increasing the number of negative votes may increase the number of posts, but it sort of races to the bottom as posters will not

      • You cannot really be active in a forum and start over as too many people will find similarities in your posting styles

        But there are so many forums - few of which are any better than any other - though some are more popular. So there would be no reason to start a new account on the same one (even if you'd got banned) you were on previously - and for forums with thousands of contributors, I doubt that anyone would notice if you did. If people really do only post for their own entertainment (which might be a more truthful reason than the conceit that they have something IMPORTANT to say) then they'd just switch to a new foru

        • I'm not a fan of the people here, I'm a fan of the conversations- that used to happen and sometimes still does happen here.

          There are other forums like you pointed out. Most of them are crap if you want to read anything half way intelligent that isn't locked in to a specific genre or area of expertise. It's certainly the reason why I've been around here for a little longer than you (actually, this is my second log in ID because I lost the password to my first years ago and had it associated with an email add

  • Comment removed (Score:3, Interesting)

    by account_deleted ( 4530225 ) on Saturday May 17, 2014 @04:42PM (#47027991)
    Comment removed based on user account deletion

  • Depends a lot on the "negative" feedback (Score:4, Insightful)

    by seebs ( 15766 ) on Saturday May 17, 2014 @05:08PM (#47028103) Homepage

    People who are trying to get "negative" responses are not getting negative conditioning, they're getting what they want.

    The trick is to give them feedback they don't want, not necessarily obviously "negative" feedback.

  • What is upvote/downvote really for? (Score:3)

    by Vellmont ( 569020 ) on Saturday May 17, 2014 @05:25PM (#47028195) Homepage

    I've never thought it was supposed to promote one kind of behavior or another. Upvote/Downvote is a means to improve signal/noise ratio, and make it possible for tens of thousands of people to communicate. It's a form of moderating, and frankly that's how it's always been. That's how slashdot was designed, and why we call it moderating, not "social conditioning". It works relatively well for what it's supposed to and certainly better than nothing at all (though I prefer reddits moderation system where there's not a limit of 5 to a post, and everyone can moderate all the time). I've never heard anyone express the idea it's a form of conditioning.

    To me the idea that receiving attention (no matter if it's good or bad) is encouraging behavior, while being ignored discourages behavior isn't all that surprising. We're social creatures that evolved in groups of 150. Being "cast out" of the group is the ultimate in shame. People have used ignoring others as a form of punishment for a LONG time. Hell, that's what a kill list was for way back in the 90s on Usenet. That's exactly what the Amish do via shunning when they want to control peoples behaviour. It's the same with other social species like dogs as well. If your dog bites you for instance, the best thing to do is to ignore it for several days. Don't look at it, act like the dog doesn't exist. When it's time to feed the dog, have someone else from outside your house feed the dog. Dogs DO NOT want to be outside the pack. If you punish the dog, you're really just engaging it and playing a dominance game. If you simply ignore it and make the dog think it's no longer in the pack... it'll get the message. Being outside the pack= death. The same is true in human interaction as well.

  • Oh, like Pharyngula.

  • My two cents (Score:4, Interesting)

    by hduff ( 570443 ) <hoytduff@[ ]il.com ['gma' in gap]> on Saturday May 17, 2014 @06:04PM (#47028387) Homepage Journal

    My thoughts are that posting in on-line communities is done mostly for reasons of self-esteem (although there are obviously other motivations) by people whose task is to share and receive useful-to-them information.

    If your self-esteem is high, the post itself provides the validation and positive or negative comments have little to no effect on what you post since validation is intrinsic.

    If your self-esteem is low, validation comes through feedback. Positive feedback is then seen to come from kindred souls and negative feedback from trolls. In both cases, validation is extrinsic and therefore has a volatile effect on the poster.

    My problem with TFA is what they quantify as "better" content. People post using words, phrases and grammar that they come equipped with; their level of education is fixed for the most part; their real-life experience and socialization is essentially fixed for the short run. Their ideas and opinions are already formed. There will not be any substantial improvement in the quality of what people post, no matter what the feedback is.

    Obviously, we need to fund more studies, especially studies done at exotic locales and funded by government money.

  • The key is that not all feedback is directed at the author. As in the case of slashdot one of the biggest benefits of feedback is to clean out the cruft. Without this slashdot would be one big "make money fast, lose weight, and invest in Nigeria" forums.

    Also the question isn't always one of looking at the authors as an average. I suspect that many authors are able to use any feedback quite nicely. If you read the comments on New Scientist (which I love) the comments are pretty much useless. But in some fo

  • Social networks with upvotes but no downvotes end up dominated by fluff, spam, and douchiness.

  • Not only do authors of negatively-evaluated content contribute more but their future posts are of lower quality and are perceived by the community as such.

    By reading Bennett Haselton stories?

    BAM! Nailed it!

    On a less snarky note: I've tried a number of times over the years to google up the study that I'm pretty sure corresponds to the following assertion, and failed. (Sources welcome.)

    Anyway, the (possibly imagined) study claimed that the best way to motivate people was to reward them *randomly*. In t

    • It's not a study - but what you're describing is the proper real-world term Karma. The poor and downtrodden accept their place, and do nothing to improve life- because they deserve it (from a past life's mistakes). See also the caste system.

  • Idea: Mod-down costs more points..? (Score:5, Interesting)

    by Vegan Cyclist ( 1650427 ) on Saturday May 17, 2014 @08:28PM (#47029077) Homepage
    It seems the down-mod still has some use, but the up-mod is preferred...how about if it just 'cost more' to mod-down posts?
  • If I were to make any single major addition to the current moderation system today, it would simply be: "You must post a comment before you may upvote or downvote any posts." Otherwise, aside from moderators being able to target and harass a single (or multiple) user's posts with downvotes with no restriction other than their point pool total, I think /. has a pretty decent moderation system going for it.

  • Works on World of Warcraft chat .. and presumably on Internet forums as well.

Slashdot Top Deals

No spitting on the Bus! Thank you, The Mgt.

Close