Facebook's 'Rosetta' System Helps the Company Understand Text Within Image, Which is Crucial In Handling Memes, Flagging Abusing Content

Facebook announced on Tuesday a new AI system, codenamed "Rosetta," which helps teams at the company as well as those at Instagram identify text within images to better understand what their subject is and more easily classify them for search or to flag abusive content. From a report: It's not all memes; the tool scans over a billion images and video frames daily across multiple languages in real time, according to a company blog post. Rosetta makes use of recent advances in optical character recognition (OCR) to first scan an image and detect text that is present, at which point the characters are placed inside a bounding box that is then analyzed by convolutional neural nets that try to recognize the characters and determine what's being communicated. This technology has been in practice for a while -- Facebook has been working with OCR since 2015 -- but implementing this across the company's vast networks provides a crazy degree of scale that motivated the company to develop some new strategies around character detection and recognition.

crucial at suppressing speech

[Anonymous Coward]

It will be used to suppress speech.

Re:

[AmiMoJo]

AC, what is your solution?

Abusive Content?

[Jarwulf]

How is content ie simple information/knowledge abusive? Does it come out of the screen and berate you?

Re:

[elrous0]

Haven't you heard? Words are now considered violence.

https://www.nationalreview.com... [nationalreview.com]

Re:

[Anonymous Coward]

Think back to the very worst moment of your life.

Now imagine that somebody recorded a video of it and posted it to Facebook.

To me, that video is "simple information".
To you, the video is not just "simple information".

Should the video be taken down?
Let's consult the Platinum Rule: Treat others the way they want to be treated
My answer is your answer to the question: Do you want the video of the worst moment of your life taken down?

Re:

[butchersong]

No. That seems very silly to me. What if the worst moment of my life was the towers coming down in NY? Does that mean that no one gets to post videos of that day? I may be able to get behind owning images of myself and taking down videos of me personally (though I doubt it) but I don't get to own and suppress ideas.

Re:

[Calydor]

To follow your tangent, do you want the video of your kid dying as the tower collapses taken down?

Re:

[butchersong]

The AC didn't specify that the video had to include an image of me personally or one of my loved ones. That is a narrower argument but doesn't address the story posted for Facebook. We're talking about "hurtful" content. I may feel some personal attachment to a certain idea. Say my parents died in a camp in WW2 or something of that sort. That doesn't mean I should be able to suppress holocaust denial. Say instead that someone was posting pictures of my dead parents in the camp, in that case I can see a

Re:

[Jarwulf]

The vast majority of 'abusive' content is not that. The OP implies they are deleting speech. The majority of 'abusive' video examples are amateur porn girls freely distributed to one or more people then suddenly decided they wanted to take back. The very same people that want to impose 'right to be forgotten porno edition' as a felony matter also say that porno and the human body is no big deal and liberating. So logically circulating amateur porn is no different from anything else and if amateur porn has a

Re:

[Jarwulf]

sorry should have added 'along the lines of what you describe'. 'Abusive' video content is more likely to be stuff like anyone proclaiming a conservative opinion.

Translation

[Anonymous Coward]

"We can now identify conservative and Trump-supporting users much more rapidly in order to ban them for #WrongThink!"

Re:

[Anonymous Coward]

fat cunts, faggy jews and ugly minorities

Which job you want? you want to fire up the ovens or shovel coals into the furnace?

LOL. Just license the Battlefield V chatbot

[elrous0]

You know, the one that labeled "white man" as hate speech.

https://www.forbes.com/sites/e... [forbes.com]

Ray Ban?

[aitikin]

Yet I always see the same Ray-Ban "Sale" spam image/post from hacked accounts...you'd think they'd train it to remove that...

Good to know

[SuperKendall]

Pretty easy to include some nonsense words that would get parsed by the AI but not by the viewer...

Used to filter ads

[Dan East]

Of course this tech is being spun to "save the children", but it is also used to screen all advertisements that run on FB. They do not want ads to contain much text - less than 20% of the area of the ad image can be text. This is detected automatically using the technology described, and their system will stop the ad if it doesn't meet that requirement.

We've found that images with less than 20% text perform better.
To create a better experience for audiences and advertisers, ads that run on Facebook, Instagram and Audience Network are subject to a review process that looks at the amount of image text used in your ad. Based on this review, ads with higher amounts of image text may not be shown. Keep in mind that some ad images may qualify for an exception. For example, book covers, album covers and product images usually qualify for an exception.

https://www.facebook.com/busin... [facebook.com]

And from the blurb:

detect text that is present, at which point the characters are placed inside a bounding box

Thus the area of the bounding boxes (after performing a union) can be at most 20% of the area of the image.

Revenge time has come!

[devslash0]

Now we should start writing text on memes using Captcha fonts.

Re:

[Mal-2]

If that's what it takes to kill off Comic Sans for good, then let's get to it.

Re:

[Mal-2]

Or just spell things in 1337 or like "a møøse once bit my sister". These are still detectable, but should drift from one meme image to the next, which means that instead of one large group, they'll fall into many small groups, which should make them harder to associate. "Fly under the radar" as it were.

imperial delusions, an empire in the sand

[harvey the nerd]

That sociopath/psychopath MZ fancies himself as "Augustus", the first Roman emperor, but neglects which Augustus really corresponds better.

Hopefully, Romulus Augustus, aka Momyllus Augustulus, might be better for the position on the timeline of MZ's empire...
One of the other Augustus over several hundred years probably better reflects his pathology...

Capcha

[duke_cheetah2003]

I imagine now we'll see internet meme in those awful fonts with twisted up letters that we humans can figure out in microseconds, but a computer is completely baffled.

Use only Comic Sans

[jfdavis668]

They will never be able to detect that.

Abusive content?

[PPH]

So they've come up with a way to identify that silhouette of the guy with the big nose?

So no more

[AHuxley]

Memes? Art? Cartoons? Blasphemy?
No more freedom of speech. No right to assemble art work and publish?