Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 



Forgot your password?
typodupeerror
×
Microsoft AI Technology

Microsoft Launches Tool To Identify Child Sexual Predators in Online Chat Rooms (nbcnews.com) 91

Microsoft has developed an automated system to identify when sexual predators are trying to groom children within the chat features of video games and messaging apps, the company announced Wednesday. From a report: The tool, codenamed Project Artemis, is designed to look for patterns of communication used by predators to target children. If these patterns are detected, the system flags the conversation to a content reviewer who can determine whether to contact law enforcement. Courtney Gregoire, Microsoft's chief digital safety officer, who oversaw the project, said in a blog post that Artemis was a "significant step forward" but "by no means a panacea."

"Child sexual exploitation and abuse online and the detection of online child grooming are weighty problems," she said. "But we are not deterred by the complexity and intricacy of such issues." Microsoft has been testing Artemis on Xbox Live and the chat feature of Skype. Starting Jan. 10, it will be licensed for free to other companies through the nonprofit Thorn, which builds tools to prevent the sexual exploitation of children. The tool comes as technology companies are developing artificial intelligence programs to combat a variety of challenges posed by both the scale and the anonymity of the internet. Facebook has worked on AI to stop revenge porn, while Google has used it to find extremism on YouTube.

This discussion has been archived. No new comments can be posted.

Microsoft Launches Tool To Identify Child Sexual Predators in Online Chat Rooms

Comments Filter:
  • by XXongo ( 3986865 ) on Thursday January 09, 2020 @02:25PM (#59603956) Homepage
    Hmm. I am curious as to where they get the data to train the system.

    If they start with actual convicted criminals, and then backtrack to look at their online activities grooming their victims-- OK, that could be reasonable. But I'm not sure that there is enough of a data set here.

    Unfortunately, there is no distinction between "grooming" and "being nice to children" except for intent, and it would be undesirable to make a system that takes any attempt to be nice as being a potential pedophile.

    • by davidwr ( 791652 ) on Thursday January 09, 2020 @02:33PM (#59603992) Homepage Journal

      "Unfortunately, there is no distinction between "grooming" and "being nice to children" except for intent, and it would be undesirable to make a system that takes any attempt to be nice as being a potential pedophile."

      It's been that way since at least the 1980s.

      It's one reason why many men in the USA and some similar countries don't want to be aline with an unrelated child for any length of time without some way to disprove any possible false allegation of anything that would ruin their reputation.

      • It's one reason why many men in the USA and some similar countries don't want to be aline with an unrelated child for any length of time without some way to disprove any possible false allegation of anything that would ruin their reputation.

        Because somehow in the past 20-50 years we seem to have switched from "innocent until proven guilty" to "guilty no matter what."

        On *ANY* topic, about anything.

        Mull that over while you're in the voting booth in November: Who keeps doing that trick? Why are they doing it?

        Vote them OUT.

        • by lucasnate1 ( 4682951 ) on Thursday January 09, 2020 @02:46PM (#59604056) Homepage

          Both sides?

          • Both sides?

            So vote them out and put fresh blood in. People who respect the Constitution and respect the rule of law.

            Oh snap, there aren't any, from any side, are they?

            Then maybe it's time people like us start running for office.

            • by SirSlud ( 67381 )

              Then maybe it's time people like us start running for office.

              Hrm, a government of crazy people? It's so stupid, it just might work!

          • by ron_ivi ( 607351 )

            Both sides?

            Right.

            So vote for a third side.

            As long as most people vote Republican or Democrat things will never improve. Every time the Republicans win and third parties get no votes, democratic strategists think "we need to be more like republicans". Every time Democrats win and third parties get no votes, republican strategists think "we need to be more like democrats".

            However if Libertarians ever got enough votes that they would have made the difference in the election; Republicans may shift their platfor

            • by Merk42 ( 1906718 )
              Vote enough Libertarians in and no one will be found guilty of child grooming.
            • by dgatwood ( 11270 )

              Unfortunately, none of the parties, including Libertarians and Greens, align very well with my views, and I don't think they really align with anyone's views. I've never met anyone who could honestly say that he or she agrees with every single position held by any major party. Yet we're forced to choose from politicians who dogmatically vote with the party on every issue. Being ruled by a larger number of groups of idiots who aren't capable of thinking for themselves is not an improvement in any meaning

          • Sounds good to me.

            I'm unaware of any solid reason to think one of the statutory duopoly parties is worse than the other on this.

            So far, I've only thought of one issue where one is worse. Well, maybe two issues.

            The difference, if any, is slight.

            Plus, they've been sharing power for over a century, with neither doing much if anything to slow it down, much less reverse it. Neither has a track record to brag about.

        • I work in education in the UK. The school I work at has policies to avoid having students alone in a room with a staff member. There must always be a witness. Sometimes the policies do get ignored, because you can't always find a third person to be present during a crisis, but we do try.

          • by ludux ( 6308946 )
            So when the policies get ignored because of an unavoidable crisis, does that mean automatic accusations of child abuse?
            • by davidwr ( 791652 )

              So when the policies get ignored because of an unavoidable crisis, does that mean automatic accusations of child abuse?

              It cuts down on the odds of a false allegation over a teacher's career significantly.

              Scenario 1: No policy. Nearly 1000 of 1000 teachers will likely find herself with a student with a motivation to lie and the ability to do so credibly several times in her career. Remember, the vast majority of students are either not motivated to lie or not convincing enough to have their allegations sustained after a professional police investigation.

              Scenario 2: Policy, but with occasional breaches: Maybe 10 or 100 of

          • Sometimes the policies do get ignored, because you can't always find a third person to be present during a crisis, but we do try.

            One way to help disprove false allegations in situations like this is to turn on a camera that makes very low-resolution black-and-white videos with sampling maybe 2 or 3 times a second.

            This will only be able to prove that two people were or were not in very close proximity at a specific time. This way, if Johnny claims Mrs. Jones touched him inappropriately and Mrs. Jones wasn't closer than a half a meter from Johnny at any given time, she can prove it.

            This won't help in "no witness available" situations

            • We do have camera coverage in all of the corridors, quite extensively - though the students long ago worked out where to find blind spots for a quick fag. There are no cameras in the classrooms though.

            • One way to help disprove false allegations in situations like this is to turn on a camera that makes very low-resolution black-and-white videos with sampling maybe 2 or 3 times a second.

              ... and this is why, for an example, increasing numbers of restaurants and bars have all members of staff wearing a body camera at all times. Different crimes being alleged, but the same response.

              The bar I was in last night probably spent several thousand pounds on body cameras. That would be less than half a week of the (i

        • Timeline corresponds to political evolutions:
          1) The republican party being hijacked by the religious right morphing into the New Conservative Fascist Party.
          2) The democratic party selling out the working and middle classes chasing corporate oligarchy payola.
          The right and left are just the 2 wings of the same sh!t bird.
      • It turns out there's a lot of child sex out there. We (the United States) have been flailing around like blind idiots trying to stop it, but nothing really seems to work. It's endemic to multiple cultures worldwide. We're still coming to terms with the amount of psychological damage sexual activity can do to minors when adults are involved. Many sexual predators were themselves victims of sexual predation as kids.

        Nobody knows what to do. People with power and money seem to delight in raping children, b

        • Jeffrey Epstein. I think you mean _Jeffrey_ Epstein.

          Theo Epstein is currently an executive with the Chicago Cubs Major League Baseball team and, as far as I know, has any accusations against him about being a child predator.

          • Jeffrey Epstein. I think you mean _Jeffrey_ Epstein.

            Theo Epstein is currently an executive with the Chicago Cubs Major League Baseball team and, as far as I know, has any accusations against him about being a child predator.

            s/has/doesn't have any/

          • Oh yeah, sorry. Wrong Epstein!

        • by lgw ( 121541 )

          . And the courts, and the legislative bodies that cater to the apprehensions of conservative voters in particular.

          When did "being against kiddy fiddling" become a conservative value? Or, if you're talking about the witchhunt over-reaction, that's never been one political side or the other.

          • When did "being against kiddy fiddling" become a conservative value? Or, if you're talking about the witchhunt over-reaction, that's never been one political side or the other.

            I can't speak for the author of the post you are replying to [slashdot.org] but the ways in which people react to these crimes can vary by political philosophy.

            Take a child rapist, and by that I mean a clear-cut, not close-in-age, no plausible "but they were in love" excuse predator situation that would be rape even if both parties were adults.

            Conservatives will say "if we can't execute him, lock the sub-human animal up in a dark hole for the rest of his miserable life."

            Liberals will say "lock him up for 30-60 years and m

            • It isn't just sentencing. Look at who pushes for the toughest sex offender registry laws. And by "toughest", I (sometimes) mean "least rational".

              Florida may be the worst registry state, but Georgia comes in pretty close. In some parts of Georgia, there are so many restrictions on where registered sex offenders - people who have ALREADY served their prison terms - can live that they can't live anywhere except under a bridge:

              https://www.timesfreepress.com... [timesfreepress.com]

              How does that even make any sense? It's like the

          • Go to rural Red-as-Blood NW Georgia and find out for yourself. Nothing scares them more than child sex. Nothing.

      • It's one reason why many men in the USA and some similar countries don't want to be aline with an unrelated child

        Women also sexually assault minors. It's not as common as men doing it, but it is very far from zero. Maybe 10 to 20% of cases that go to court.

        There have been several in the press here in the last week or so.

    • by jm007 ( 746228 )
      damn, you took all my talking points ;-)

      and from their legal dept's standpoint, there'd be a shitton of procedures in place before casting aspersions; there should be strong penalties for false positives but sometimes that's not how life works; would suck to be collateral damage on the way to 'saving the children'
      • I believe that's the purpose of flagging the user for a human review as described. There's a big difference between being nice to a kid, and being nice, then asking for location info, pictures, etc which could be grooming behavior.

        Here's an example. Earlier today I was on a python forum and a kid was posting about learning Dijkstra's algorithm. So I sent the kid a message praising him for tackling it (because after learning it like 10x, it just won't stick with me). I told him/her that if they really lik
        • by sinij ( 911942 )
          Human review is expensive. We know that many social networks (e.g. Youtube) use algorithmic bans. There is no reason to belive this is going to be any different. Good luck appealing the ban.

          Also, imagine if kid was asking about fsck and umount, you would be in jail by now.
          • Well, considering the article specifically cites using a human reviewer, I'm not too worried. If it ends up being used algorithmically, ok, I'd cede your point. But as of right now that is not the project as it's defined. As for appealing a ban, I'm not that worried about that as fall out. I have no fundamental "right" to access another person or company's systems. If they flag me and choose to remove me, that's their right to do so.
            • by sinij ( 911942 )

              As for appealing a ban ... I have no fundamental "right" to access another person or company's systems.

              A rational actor under such regime would not talk online to any child for any reason under any circumstances.

      • Easy, bribe the politicians to enact laws making this thing immune from libel suits.

        The bribe will probably involve making sure this thing can not target politicians.

      • by DRJlaw ( 946416 )

        and from their legal dept's standpoint, there'd be a shitton of procedures in place before casting aspersions; there should be strong penalties for false positives but sometimes that's not how life works

        That's not how the law works either. Expressing an opinion based upon disclosed facts, no matter how wrong that opinoin is, is expression protected by the first amendment.

        If your aspersion is "I think this person is grooming this child and here are the logs that were flagged," then of course you can still b

        • by jm007 ( 746228 )
          yeah, my layman's terms makes my point a bit slippery; what I'm trying to figure out is once something is flagged and run by a human inspector, who do they tell? and how does that get done w/out exposing themselves to liablitity? or, if I understand you correctly, they're not really exposed at all, no?

          might be another over-hyped "AI will save the world" thing, too, and useful for nothing but marketing

          also, this one: "Defamation requires a false statement of fact, as opposed to an incorrect opinion
          • by DRJlaw ( 946416 )

            how is it decided which is which? is it like "I BELIEVE you're a pedo" vs "You ARE a pedo" ? or probably more sophisticated?

            The wording of the statement does not make nearly as much of a difference as non-lawyers believe. Hedging everything as a belief or opinion will not necessarily protect you from liability for defamation. The key is that both of those statements are opinions/conclusions suggesting undisclosed facts [washingtonpost.com]. That is different than an opinion/conclusion drawn from disclosed facts [rcfp.org], where the re

            • by jm007 ( 746228 )
              great links, been digesting them, thanks; are you Eugene?

              libel per se is interesting; I could get lost going down every rabbit hole

              thanks again
        • by lgw ( 121541 )

          If you present yourself as an expert, you can be sued (and lose) for your expert opinion. That seem relevant here. But it's only relevant if there's some public disclosure. But that never happens with controversial figures, so no problem at all ...

          • by DRJlaw ( 946416 )

            If you present yourself as an expert, you can be sued (and lose) for your expert opinion. That seem relevant here.

            Ummm, no. Joe Random cannot sue an expert for their expert opinion. If you hire an expert for their opinion, that's a whole different story, but also a whole different situation from "can I be sued for my process of identifying and reporting potential child molesters."

            • by lgw ( 121541 )

              I say something public and defamatory, but I say "in my opinion", it's damned herd to recover damages. That changes if I present myself as an expert and say "in my opinion as an expert on X". That's claiming a fact, or very close to it. Of course, truth is always a defense in the US, but if you happen to be wrong, you're asking for trouble.

              The "process of identifying and reporting potential child molesters" isn't in itself much of a cause for a legal action, but if it's done poorly, or it's made public,

              • by DRJlaw ( 946416 )

                I say something public and defamatory, but I say "in my opinion", it's damned herd to recover damages. That changes if I present myself as an expert and say "in my opinion as an expert on X". That's claiming a fact, or very close to it.

                "That's Not How It Works. That's Not How Any Of This Works."

                Saying "in my opinion" won't save you, being an expert saying "in my opinion" isn't going to change the outcome from one in which a non-expert says "in my opinion," and saying "in my opinion as an expert" cannot chan

                • by lgw ( 121541 )

                  What nation are you talking about? In the US, it's nearly impossible to win damages over any defamatory statement unless a reasonable person would believe it to be a statement of fact, not opinion. That's because truth is a very good defense, and however outrageous my statement of opinion about you might be, it's damn hard to contest that it is truthfully my opinion.

                  However, that's not such a good defense if you're presenting an expert opinion as a professional in some field, but no expert in that field w

                  • by DRJlaw ( 946416 )

                    What nation are you talking about? In the US...

                    The US. The same one that the many sources that I linked to apply to [slashdot.org].

                    And in my expert opinion, you don't have a clue as to what you're talking about. Feel free to provide your sources. Or to sue me. You'll fail either way.

                    • by lgw ( 121541 )

                      I see. You're just too dumb to draw the connection between being an expert, and having knowledge of "undisclosed facts" that the non-expert wouldn't recognize. Quite dumb, then.

                    • by DRJlaw ( 946416 )

                      You're just too dumb to draw the connection between being an expert, and having knowledge of "undisclosed facts" that the non-expert wouldn't recognize.

                      That's quite an attempt at a walk-back, and an illogical one at that. Having knowledge of "undisclosed facts" cannot make you liable for defamation.

                      Stating an opinion based upon disclosed and true facts, without explicitly or implicitly suggesting that the opinion relies upon undisclosed and defamatory facts, remains an opinion whether you're a layperson or

    • Hmm. I am curious as to where they get the data to train the system.

      If they start with actual convicted criminals, and then backtrack to look at their online activities grooming their victims-- OK, that could be reasonable. But I'm not sure that there is enough of a data set here.

      Unfortunately, there is no distinction between "grooming" and "being nice to children" except for intent, and it would be undesirable to make a system that takes any attempt to be nice as being a potential pedophile.

      I think they forced the AI to watch old episodes of To Catch a Predator with Chris Hansen.

    • Assuming the 'predator' even knows they are talking to a minor. It's not at all clear online. The same words which could be considered 'grooming' a child are just flirting when said between two adults.

    • by Sloppy ( 14984 )

      Keywords: cockpit, naked, Turkish prison, gladiators

    • by spitzig ( 73300 )

      I recently read an article about a group of adults pretending to be kids online to catch pedophiles. They might use conversations like that to catch them.

        It's more than just being nice. If you ask how old someone is, then start a conversation about sex or ask for naked pics, that's not just being nice.

    • by AHuxley ( 892839 )
      1. Learn slang, region ways of "chatting" as needed. Set with a GUI for "state" as location for any state law enforcement as needed.
      All over the USA for federal... city/state for city police.

      2. Start with opening chat lines using the AI.
      3. The criminal keeps on chatting... thats a crime.
      4. The criminal drops that chat quickly ... that is a criminal looking for a different chat.
      5. All use of a VPN is criminal. Not using a VPN? A criminal trying to be an "average" ISP user.

      100% success rate.
    • It's one of the problems faced by many men going into teaching, especially those who love working with kids in the early stages of education system such as kindergarten and primary school levels. My daughter's first two teachers when she was around 4 years old were men at the local school. She had a great time and the teachers had a huge amount of respect. However you ask any of the parents, myself included and I consider myself very open minded, and there's a tiny, tiny, tiny seed of doubt sown by our para

    • by AmiMoJo ( 196126 )

      Probably from reported and investigated incidents. It's extremely common on some platforms. Try creating an account pretending to be a child, it's insane.

    • it will use blockchain in its organic development of understanding panty-sneakers ... trained by the internet ofc. since the internet invented pedophiles (and drugs ...) and Al Capone got rich on crypto just like Escobar , the unabomber would certainly have been caught as fast as obama bin laden if they had this type of supreme microsoft programming A.I. so 'im like "cant even code an OS that's more functional than an extinct dinosaur(and twice as heavy)" "will not get into automatic identification and
  • Bot 1 pretends to be a child to lure predators so some TV show can sting them.

    Bot 2 pretends to be a predator so some TV show can show scared parents how easy it is to trick kids.

    Bot 3 is run by BigDataCorp and keeps flagging bots like Bot 2 resulting in a lot of wasted time by BigDataCorp and possibly law enforcement.

    Meanwhile real 12 year olds are laughing their NSFW parts off sharing screen grabs and memes of all of this over on randomnumberchan.

  • Sounds like a good reason to never, ever enable chat in xbox live or use skype with anyone you don't already know.

    Unless you're Paul Krugman, you can't just walk back accusations of being a nonce.

  • I don't understand why they don't just age-lock rooms. Under 20, over 20, -- something like that.

  • by sinij ( 911942 ) on Thursday January 09, 2020 @02:52PM (#59604082)
    If government or large corporation tells you some action is about catching pedophiles, you can be all but certain that the real reason is violating your privacy. Here is they telling you exactly what they are doing: "Microsoft has been testing data-mining on Xbox Live and the chat feature of Skype".
    • "Every tool and power you give me to protect you from your enemies and the things you hate, will also be used to make you my bitches!"

      ~Government

    • by DogDude ( 805747 )
      If government or large corporation tells you some action is about catching pedophiles, you can be all but certain that the real reason is violating your privacy.

      So then, are you proposing that our government stops trying to catch pedophiles?
      • by gweihir ( 88907 )

        That "counter-argument" does not work on so many levels it is staggering. You are just virtue-signalling without any understanding of the matter at hand at all.

        • by spitzig ( 73300 )

          His counter-argument was a better argument than yours to his. Yours was just "nuh-uh, you're just acting like you're a good person."

          • by gweihir ( 88907 )

            Nope. Try again. Hint: There was no actual argument in his statement. Also, you urgently need to look up "virtue signalling". It has nothing to do with being a good person.

        • by DogDude ( 805747 )
          Suggesting that we arrest pedophiles is "virtue signalling"? Jesus Christ, what's wrong with you?
    • by gweihir ( 88907 )

      Indeed. "Human review"? Far too expensive! This will never get to that stage and it is definitively not intended to.

  • You mean some marketing minion missed an opportunity to declare yet another instance of AI???
    oops, spoke to soon, found "AI" in the actual article.
  • by the_skywise ( 189793 ) on Thursday January 09, 2020 @03:05PM (#59604128)

    that all your conversations are being monitored for content. Not just words, but phrases, talking styles and what you're talking about.

    Sure... it's "for the children" but this could easily be adapted for other uses that many political regimes would like to watch for.

    • You weren't under the assumption that this was already the case? There are already government agents that pose a children who work to identify and arrest pedophiles, so it's pretty safe to assume that there was already an effort to track these individuals. Even if they weren't monitoring online chat for this kind of stuff earlier, you can bet the companies were doing it for advertising purposes and you can sure as hell bet the government was doing it to look for terrorists or some such people. If you alread
      • by mishehu ( 712452 )
        They're trying to cut down on manpower costs for this. Just imagine one day in the not distant future where a pedophile bot engages with Microsoft's pedophile-tracking bot. No humans necessary in the equation...
  • Pedo accusations are about the worst. Any 'positive' needs multiple layers of verification because this kind of accusation doesn't wash off.
    • by gweihir ( 88907 )

      On the plus side, if MS ever gets into the business of destroying reputations, they will now have a lot of material they can use! And who cares whether it is true or not.

  • Its all just dudes and FBI agents. This is all just kabuki
  • This here is like putting another layer *on top* of the trainwreck that is machine translation. And then fucking attaching it to a machine to ruin people's lives! Using the tried and true methods of denunciation, popular among oppressive regimes, and a level of "If you give me seven lines...", that would Cardinal Richelieu cum (on a child of course, as it was his "tradition").

    Call me when they can even get basic sarcasm right. (Which is impossible, as the whole point is that it plays right on the precise e

  • What are you doing here?

  • Once this gets out there, where ever 'there' is, I can see this as a tool used by predators to weed out cops (or Chris Hansen) posing as under age kids to round up the pervs for a meetup.

  • Just so you know, ASL is commonly referred to as American Sign Language. And about 8% of the US population knows sign language. And it's prevalent on Twitter as well. I hope Microsoft doesn't get confused with pedos' future attempts to evade through immersion into the Deaf crowd by using the shortened version of A/S/L (age, sex, location) that I often read about in social media.
  • Has the catholic church already banned Microsoft and declared it to be in league with the devil?

It is not best to swap horses while crossing the river. -- Abraham Lincoln

Working...