Please create an account to participate in the Slashdot moderation system

 



Forgot your password?
typodupeerror
×
Google The Internet

Google Releases an AI Tool For Publishers To Spot and Weed Out Toxic Comments (bbc.com) 195

Google today launched a new technology to help news organizations and online platforms identify and swiftly remove abusive comments on their websites. The technology, called Perspective, will review comments and score them based on how similar they are to comments people said were "toxic" or likely to make them leave a conversation. From a report on BBC: The search giant has developed something called Perspective, which it describes as a technology that uses machine learning to identify problematic comments. The software has been developed by Jigsaw, a division of Google with a mission to tackle online security dangers such as extremism and cyberbullying. The system learns by seeing how thousands of online conversations have been moderated and then scores new comments by assessing how "toxic" they are and whether similar language had led other people to leave conversations. What it's doing is trying to improve the quality of debate and make sure people aren't put off from joining in.
This discussion has been archived. No new comments can be posted.

Google Releases an AI Tool For Publishers To Spot and Weed Out Toxic Comments

Comments Filter:
  • Good news! (Score:5, Funny)

    by Anonymous Coward on Thursday February 23, 2017 @09:05AM (#53916981)

    So all those posts from SJWs can easily be removed. I'm a fan of this.

  • by Anonymous Coward

    Let's automate oppression!

  • by Mashiki ( 184564 ) <mashiki.gmail@com> on Thursday February 23, 2017 @09:14AM (#53917041) Homepage

    Well, let's all bow down the moral arbiters of justice then. I'm sure that they'll be right on top of removing speech they disagree with. Then moving onto the useful idiots that cheered this on in the first place.

    If you're willing to remove some speech because it makes you upset, there's nothing stopping others from doing the same to you later.

    • Re: (Score:2, Insightful)

      If you don't like an online forum's moderation, then you are completely free to find one more to your liking, or even start your own.

      But believe me, sooner or later if your online forum takes off in any way, you're going to get some trolls, and if they aren't checked, they'll drive out anyone reasonable people. I've seen more than one forum collapse under the weight of uncontrolled trolling. It even happened to a local community web forum, where three or four very abusive posters who seemed to have infinite

      • by Mashiki ( 184564 ) <mashiki.gmail@com> on Thursday February 23, 2017 @10:00AM (#53917357) Homepage

        Yeah here's the problem. Let's look at a site like neogaf, ever wonder why at one time it was the place to go and developers would post there and people would leak information. And now developers don't? Active participation is down? It was the moderation as you pointed out, but let's look at their definition of a troll, which basically boils down to "anyone who doesn't subscribe to the narrative." Let's look now, at what will get you banned. Have a contrary view of feminism? Banned. Support some ideas of a MRA? Banned. Have differing view points on global warming? Banned. Prefer the xbox vs playstation? Likely banned. Don't like your vidya characters to look like they were hit with a bat? Banned.

        The problem is in many cases, moderators especially in this day and age use that as a form of power projection. A good example of this is /r/politics or /r/canadapolitics where you have moderators who ban people for pointing out factual information because it goes against the prevailing group think. Have a nice article about how forums become infiltrated by people pushing authoritarian viewpoints. [oneangrygamer.net]

        I ran a BBS in my teens, and it got large enough that I was considering applying for a regional fido:net hub. At nearly 700 people and 4 nodes, I picked moderators for my forums who weren't assholes, who usually worked, and if they stepped over the line they were given one warning then booted. As sysop, I expected my mods to be impartial. And if a friend was involved to pass the issue to another mod to deal with. I ran on Renegade.

        • No, that's not the definition of a troll. FRom the earliest days, trolls were people who said inflammatory things, started flamewars, and generally were bullying and derogatory, and even in the early days of Usenet, moderated groups would see such people banned. There are ways to express contrary views that doesn't involve threats and bullying.

          • There are ways to express contrary views that doesn't involve threats and bullying.

            Funny, because I heard from many people that there are ideas that should never be expressed. Leftists told me that any research that shows a difference in mental capacity between people of different genetics should be forbidden even if truth, rightists (I'm Israeli) told me that any research that shows that there is no single jewish ethnicity is antisemitic and therefore automatically forbidden. Was there ever any time in history where your quote above was actually correctly?

          • by Mashiki ( 184564 )

            Guess you haven't kept up with what's been going on for the last ~10 years. Because the old definition of trolling has long gone out the window, and trolling is basically considered "anyone who disagrees with me" or "has a viewpoint contrary to my world view." You enjoying the era where social justice reinvents something or changes the definitions of words in order to self-victimize yet?

            It's kinda like the casual use of sexism, or racism. As a response to anything, especially by those who are heavily int

            • by AmiMoJo ( 196126 )

              "trolling is basically considered "anyone who disagrees with me"

              See my signature.

              • by Mashiki ( 184564 )

                You signature makes no sense. Since anyone can go find a person who supports social justice, and has a pro-authoritarian slant to boot. In turn, you can find a few people who post here and support said social justice values and also have a pro-authoritarian slant.

        • by AmiMoJo ( 196126 )

          At the risk of sounding like a broken record, where is your evidence of this?

          Because every time in the past when we have actually looked into these claims, it's just trolls trying to hide behind "my views are just conservative, I'm not trolling!" and no-one falling for it.

          • by Mashiki ( 184564 )

            At the risk of sounding like a broken record, where is your evidence of this?

            Neogaf isn't a hive mind? [imgur.com] Or will you claim that *insert thing isn't a source* like you usually do? Feel free to hit your favorite search engine and use "neogaf" "ban". Sites like ceddit/go1dfish.me [ceddit.com] do a good job of covering the deletion of things. Subs like /r/subredditcancer [reddit.com] show individual case-by-case examples. You can now claim that these show nothing, and there are no problems like you usually do.

    • Re: (Score:2, Troll)

      by T.E.D. ( 34228 )

      Well, let's all bow down the moral arbiters of justice then. I'm sure that they'll be right on top of removing speech they disagree with.

      This is precisely what the spammers warned us about when the exact same Bayesian [wikipedia.org] techniques were applied to get rid of spam. But you didn't listen to their plight, and now here we are having abusive trolls content removed before too. Where will the carnage end?

      • And now I have to check multiple folders when I'm expecting an email instead of just one... Password reset? Let's see, will gmail automatically put that in inbox? (unlikely) How about Spam? Maybe. "Social"? Maybe. Not come at all because that same email text is sent to multiple people? Maybe. Who knows.
    • by arth1 ( 260657 )

      Well, let's all bow down the moral arbiters of justice then. I'm sure that they'll be right on top of removing speech they disagree with.

      Never attribute to opinion that which can be adequately explained by greed.

      They may agree fully with what you said, but a comment that a company's customer support sucks is still going to be removed.
      Preferably automatically, so they won't have to pay an outsourced minimum wage slave to do the job.

  • Did Donald Trump just take over the PR role for Google?

    With YouTube, an Alphabet/Google Company, having the cesspool of toxic comments that it has, I don't see how anyone at Google could claim with a straight face that Google's infrastructure is any good at filtering out toxic comments.

    • by Anonymous Coward

      "The system learns by seeing how thousands of online conversations have been moderated and then scores new comments by assessing how "toxic" they are and whether similar language had led other people to leave conversations."

      We all know that people abuse the moderation system to equate "I disagree" with modding a post down. If I had to guess, I'd say that what constitutes a "toxic" comment will be a mix between: a) toxic and b) unpopular opinions.

      • Doesn't even have to be "unpopular." "I support Trump" and "I support Hillary" were both widely held, popular opinions. But you damn well know people are moderating those comments based on their own political bent and not on "toxicity."

        This is why reddit is such garbage. Yes, the rules explicitly state that up votes and down votes mean "contributes" or "doesn't contribute" and not "agree" or "disagree." So on any controversial topic what you should find at the top should be the very best, most well thought-

    • Re: (Score:2, Insightful)

      by Mashiki ( 184564 )

      This might be new to you, but the left have been pushing for speech controls for years at google. It's not Trump, it's not those on the right. The right are the ones defending free speech, and it's the left who are trying to censor it. Whether it be no-platforming, violent assaults on people, using bomb threats or other tactics to shut down venues. The social justice brigade [thegg.net] has been doing this for a long time, it's why github is such a steaming pile of shit now. You can see this when companies start i

      • Your comment might sound toxic to some people. Your first few sentences can easily be read that people on the left are pro censorship and anti free speech. If you actually believe that, then no point reading any further.

        There is nothing wrong with intelligent people having a civil discussion about public policy, even if they have disagreeing views.

        It is the name calling and suggestions that MY SIDE is 100% Correct and on the side of the angels, and YOUR SIDE is 100 % Wrong and on the side of the Dev
        • by Mashiki ( 184564 )

          The reality is, if someone considers my comment "toxic" because it hurts their feelings. They probably need to spend some time outside of their social bubbles and realize that the world isn't a happy place where people give you what you want if you cry loud enough. The sad thing is, it *is* the left(not all of it), that are pro-censorship and anti-free speech. It's the same situation that we saw back in the 80's and 90's. Remember? When you had the religious nuts holding power in quite a few places, cl

      • by AmiMoJo ( 196126 )

        Trump has an even better plan. Don't bother censoring everyone else, just call them liars and dishonest until people only listen to you. It's the same strategy used by cult leaders everywhere, except instead of a compound where he can brainwash his followers he has Twitter.

        • by Mashiki ( 184564 )

          Trump, is simply vocalizing what a lot of people already think on the state of the media. And him vocalizing what people already believe is one of the big reasons why he was elected, round that out with the out-right refusal to bow to political correctness. Why do you think trust of it is so low in western countries? It's not because of Trump. It's because of the media.

  • by Anonymous Coward

    . . . onto all the ways we can fool this "ai" to get snide comments through
    . . . cry for all the false positives there will be
    . . . all the tricky cases, such as someone quoting "a toxic post" as an example of how not to do it . . .

  • Awesome (Score:5, Insightful)

    by Unknown User ( 4795349 ) on Thursday February 23, 2017 @09:24AM (#53917101)
    Great for our echo chambers, so we can hear more from people whose opinions we already have. What could go wrong with that? /s
    • Re: (Score:2, Troll)

      Ideas are one thing. Abusive trolls is another.

    • Comment removed based on user account deletion
    • It is entirely possible to have a completely contrary position to someone and express it without being toxic.

  • It smacks of censorship but perhaps it could be used to prioritize important complaints.
    • by s.petry ( 762400 ) on Thursday February 23, 2017 @09:33AM (#53917167)

      Who will be filing the most complaints? The people with money to pay astroturfers and sockpuppets? That is what we have everywhere else, so why would Google's app be any different?

      What will the complaints be about? Same thing we see everywhere else, which is anything not pro communist/extreme leftist?

      This is a promotion of fascism, not freedom.

  • by Baron_Yam ( 643147 ) on Thursday February 23, 2017 @09:27AM (#53917127)

    >The technology, called Perspective, will review comments and score them based on how similar they are to comments people said were "toxic" or likely to make them leave a conversation.

    Experience shows that toxic comments encourage participation as they simultaneously reduce participant satisfaction.

    You want customers hitting F5 and (hopefully) seeing more ads on your site? Get people's egos involved and get them competing and hating on each other.

    • by AmiMoJo ( 196126 )

      Experience shows that toxic comments encourage participation as they simultaneously reduce participant satisfaction.

      In other words, trolling works.

  • Comment removed (Score:5, Interesting)

    by account_deleted ( 4530225 ) on Thursday February 23, 2017 @09:30AM (#53917141)
    Comment removed based on user account deletion
    • by Anonymous Coward

      Slashdot has moderation that helps filter out the worst trolls, but it also amplifies the groupthink which is commonplace on the Internet. Just go to any Slashdot thread on H1B or copyright enforcement and see how many "pro" posts get modded up. And the articles on female coders seem to bring boys here back to their days of sitting in the back row of the middle school classroom.

      • Of course you get a bit of group think but for goodness sake, don't you mostly find it funny? H1B is mostly rated poorly by a group of people who more than likely have lost work because of H1B? And of course the capitalist businesses that hire the H1B would think it the best thing since sliced bread because it lowers costs. Do you think we are idiots who do not know that? FFS stop being such a snowflake. There are plenty of other places where a different partisan view is expressed and if you are not aware o

    • by AmiMoJo ( 196126 )

      It's about finding an appropriate place to shitpost. That place is 4chan, and the many other shitposting speciality sites.

      On the other hand you have sites like Github, which are about collaborating to write software. They probably don't want too much trolling. Needs to be mostly safe-for-work because their business model requires it. So they have an interest in nuking toxic users.

      Feel free to argue that Github should change policy, or start your own version (didn't 4chan try that?) and see if it becomes pop

  • And we will define "toxic" as anything that doesn't support authoritarian technocracy.

    • And we will define "toxic" as anything that doesn't support authoritarian technocracy.

      Who is "we"? If you run a website you can use this tool to make it easier to remove posts you don't like. That's up to you. If people don't like it your site will suffer, but that's all it is. It's not some government control, and it's not something you can't already do on your own.

  • OK, my bet is they are using tensorflow or another neural network behind this with supervised training. NNs are great at performing classifications by basically correlating everything to everything, but they really rely on valid output data to match the input data. Saying some audience rated the post as 'toxic' is really a bad methodology of training such a network. They will likely have a ton of bias based on the audience they have, such that in a democratic website, probably anything that was said by t
  • by geekmux ( 1040042 ) on Thursday February 23, 2017 @10:06AM (#53917399)

    If you have a troll problem, then moderate properly by banning. Censorship is not the answer because truth will ultimately suffer.

    The definition of toxic will never be a constant, and I can already seen forums looking for revenue streams to favor those paying for certain "filters".

    • If there was such a thing as big-T Truth that was perfectly knowable, this, and other forms of censorship, would be great. Anything false could simply be censored and anything true let stand.

      It's because perfect truth is inaccessible to us that we need to embrace truth seeking rather than the truth as we know it. Rejecting censorship is part of that stance.

  • "Free Speech for Me not for Thee" I believe title of Nat Hentoff's decades old book. Which of the following is "toxic comment" Obama has spent his entire adult life as a dedicated Communist. Trump has spent his entire adult life as a dedicated Fascist. How would a person or a computer AI, determine which is a "toxic comment" ? My guess is that the organizations behind virtually every attempt at eliminating "toxic comments" or "fake news" are convinced that in fact Trump at the very l
  • by JustNiz ( 692889 ) on Thursday February 23, 2017 @12:31PM (#53918487)

    This tool sounds like every PeeCee SJW's wet dream.

  • If my "perspective" counters yours, should I have the right to remove yours?
    If an American Internet website creator allows public access without registration (newspaper, social, journal, blog, whatever) then the 1st Amendment applies and they have no right to restrict what visitors to their sites post.

    This should be obvious from the fact that a Christian baker was fined (put out of business?) for refusing to bake a homosexual, a lifestyle diametrically opposed and forbidden by the Christian faith (Lev 18:

  • by AbRASiON ( 589899 ) * on Thursday February 23, 2017 @04:32PM (#53920347) Journal

    I'm just going to say it... "Toxic" is a social justice flag word. Like over use of "gross" or "icky" or referring to people as babies "shitlords" "edgelords" or "shitbirds" and so on.

    Now that we've established potential bias here, we need to define "Toxic"? Is that simply not agreeing with the status quo? Is an opposing opinion debating a topic deemed "Toxic"?

  • By nature AI censoring (moderating or also so called recommendation systems) work on premise that only information "you/author likes" are permissible. And there is the danger because the ultimate goal is to leave only comments (ads/recommendations) that are totally in line with given topic/desire/need/view...

    Now imagine the site of the fundamental extremist - in that context AI will ban all moderate comments as they will be deemed "toxic" by extremists leaving only comments supporting the fundamentalists po

In practice, failures in system development, like unemployment in Russia, happens a lot despite official propaganda to the contrary. -- Paul Licker

Working...