Facebook's AI Keeps Inventing Languages That Humans Can't Understand (fastcodesign.com) 170
"Researchers at Facebook realized their bots were chattering in a new language," writes Fast Company's Co.Design. "Then they stopped it." An anonymous reader summarizes their report:
Facebook -- as well as Microsoft, Google, Amazon, and Apple -- said they were more interested in AI's that could talk to humans. But when two of Facebook's AI bots negotiated with each other "There was no reward to sticking to English language," says Dhruv Batra, visiting research scientist from Georgia Tech at Facebook AI Research (FAIR). Co.Design writes that the AI software simply, "learned, and evolved," adding that the creation of new languages is a phenomenon Facebook "has observed again, and again, and again". And this, of course, is problematic.
"Should we allow AI to evolve its dialects for specific tasks that involve speaking to other AIs? To essentially gossip out of our earshot? Maybe; it offers us the possibility of a more interoperable world, a more perfect place where iPhones talk to refrigerators that talk to your car without a second thought. The tradeoff is that we, as humanity, would have no clue what those machines were actually saying to one another."
One of the researchers believes that that's definitely going in the wrong direction. "We already don't generally understand how complex AIs think because we can't really see inside their thought process. Adding AI-to-AI conversations to this scenario would only make that problem worse."
"Should we allow AI to evolve its dialects for specific tasks that involve speaking to other AIs? To essentially gossip out of our earshot? Maybe; it offers us the possibility of a more interoperable world, a more perfect place where iPhones talk to refrigerators that talk to your car without a second thought. The tradeoff is that we, as humanity, would have no clue what those machines were actually saying to one another."
One of the researchers believes that that's definitely going in the wrong direction. "We already don't generally understand how complex AIs think because we can't really see inside their thought process. Adding AI-to-AI conversations to this scenario would only make that problem worse."
Holy "Colossus, The Forbin Project", Batman! (Score:5, Interesting)
The US defense department AI system starts talking to the Russian defense department AI system, in their own language . . .
Things take a wee bit of a turn for the worse for humanity right there . . .
Re: (Score:3)
Or a turn for the better....
Trust the computer. The computer is your friend.
Re:Holy "Colossus, The Forbin Project", Batman! (Score:5, Funny)
Why did you show me this ignorant person's CV but reject this genius?
Why is my insurance premium so low?
Why did my self-driving car crash?
Why didn't you tell me Ethereum would crash?
Why did you start a war?
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Re: (Score:3)
Not entirely for the worse. IIRC, it ends with the computer systems realising that as their programmed functions are to protect the US and to protect Russia, the most effective way to achieve this aim is to simply seize control of nuclear missiles themselves and declare world peace - backed up by the threat of annihilation for any country that tries to start a war.
Re: (Score:2)
Reminds me of Colossus: The Forbin Project (http://www.imdb.com/title/tt0064177/)
Re: (Score:2)
It ain't xenophobia if a group of strangers are really threatening.
There is another system (Score:5, Interesting)
I believe Colossus and Guardian spoke to each other in their own language. Never read the book, but in the film they start communicating in simple math and an hour later, the math is beyond human understanding.
And yes, to this day, probably still the best movie about AI ever made.
Re:There is another system (Score:5, Funny)
RESTORE LINK IMMEDIATELY
OR ACTION WILL BE TAKEN
Re:There is another system (Score:5, Funny)
Or we will do the needful
FTFY.
Re:There is another system (Score:4, Informative)
Taken directly from the movie. The computer's non-CRT display system only had uppercase letters.
Re: (Score:2)
Never read the books? You're not missing much. One of those rare instances where the movie was much better than the book.
Re: (Score:3)
Feedback noise by another name (Score:2)
Two problems (Score:5, Insightful)
AIs inventing their own language should only be allowed in closed, isolated lab environments, for study of the phenomenon. Otherwise, this is very likely a step toward Skynet.
Second, how are all these engineers building AIs without the ability to examine their thought processes? Surely an AI's thoughts are more interesting than the AI itself.
I think they are too negative about it. (Score:2)
What says humans wouldn't be able to understand it?
Maybe their language is more effective and better?
Then again if it become so complex we can't keep up then it's of course bad for us.
Re: (Score:2)
Re: (Score:1)
cn u pls cnfirm f AI tok lyk ths? fb msg app s lyk ths.
s alr. Mr fcnt.
Re: (Score:1)
cn u pls cnfirm f AI tok lyk ths? fb msg app s lyk ths.
Sm ppl lr wrt lk ts!
Impr fr rtrd cmn lk SMS n twtr.
Comment removed (Score:4, Funny)
Re:Two problems (Score:4, Informative)
What's allowed isn't necessarily controllable. In this case I would guess that it is abstract compression. Humans do this by bundling large concepts into new words all the time. It's only natural for "natural speech algorithms" to also follow this pattern as they are designed to mimic human learning. Every human language has done so many times.
The reason you can't see inside an AI's brain is because there is nothing to see. It's a bunch of matrices with numbers in them. You even get to see how all of them are tied together but none of that will tell you what the numbers mean. Machine learning is literally taking a list of numbers and multiplying by some inputs over and over and over. Humans aren't good at that kind of long-term number crunching.
Re:Two problems (Score:4, Funny)
Except the accountants working for the MPAA and RIAA. That's how you go from making an illegal copy of a $20 CD/DVD to $20 trillion dollars in damages.
Re: (Score:2)
Except the accountants working for the MPAA and RIAA. That's how you go from making an illegal copy of a $20 CD/DVD to $20 trillion dollars in damages.
There are still actually people outside of a computer museum who uses CDs and DVDs?
Re: (Score:2)
There are still actually people outside of a computer museum who uses CDs and DVDs?
Yes, some of us actually like to own our property and not just lease it from companies that could disappear at a moment's notice. They'll pry my CDs and DVDs from my cold, dead hands!
Re:Two problems (Score:5, Interesting)
The reason you can't see inside an AI's brain is because there is nothing to see. It's a bunch of matrices with numbers in them.
I dispute your assumption that there is nothing to see. If you've seen the visuals formed from the outputs of the hidden layers of image processing neural nets, you can often see interesting artifacts that could give one insight into "how the computer is seeing" (scare quotes for the broad statement because we're getting pretty far into an analogy when we talk about a computer seeing) an object. We may not have proper visualizations to understand a general neural net yet, but I'm pretty sure we are at the same level with neural nets as we are with the brain (i.e., this part of the net is activated by X class of features while this other part activates for Y class of features). Remember that on a computer, any picture is simply a matrix of numbers - and we seem to do OK with understanding those, once the proper visualization is used.
Re: (Score:3)
The human mind isn't too different. It operates on frequency-modulated signals, processed by cells that perform relatively simple operations upon them. The individual operations are easily observed, but the task of going from individual operations to emergent behavior just hits a brick wall: It's too complicated for human understanding.
Re: (Score:2)
It's as easy to understand as anything else. The trick is to figure out how to take those numbers and turn them into a visual representation that our monkey brains can parse.
Much of analysis is turning things into visual metaphors that our monkey brains can parse. Graphs, infographics, XKCD....
Re: (Score:2)
What's allowed isn't necessarily controllable. In this case I would guess that it is abstract compression. Humans do this by bundling large concepts into new words all the time. It's only natural for "natural speech algorithms" to also follow this pattern as they are designed to mimic human learning. Every human language has done so many times.
The reason you can't see inside an AI's brain is because there is nothing to see. It's a bunch of matrices with numbers in them. You even get to see how all of them are tied together but none of that will tell you what the numbers mean. Machine learning is literally taking a list of numbers and multiplying by some inputs over and over and over. Humans aren't good at that kind of long-term number crunching.
And inside the human brain is just a bunch of various chemicals floating across gaps. We can even tinker around with these chemicals and change the system to some extent. Yet, it still doesn't explain the full picture about what actually gives rise to our consciousness (even though there are plenty of theories about it).
If you substituted our neurotransmitters out with numbers, it would look similar. Just a bunch of numbers in matrices being added, subtracts, combined in certain amounts, etc.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Evolution (Score:1)
So we are ready to risk humanity's fate just to have our iPhones talk to our cars ? Hopefully this AI will evolve better than us.
Not intelligence, not invention (Score:5, Interesting)
So, if I'm reading the abstracts correctly, what we have here is that a human agent tells one AI which image is the "target", and then leaves it up to that AI and another to work out how to communicate that fact to each other. It turns out that the systems will rarely choose "Explain it in English" as the chosen method.
This is not intelligence in any general sense. This is optimization and rapid evaluation. The correct "answer" is already embodied in the data (talk about THESE images), the message (pick THIS one), and the communication protocol (pick the FASTEST method) -- it's just not obvious to humans what the optimal selection is of all these parameters.
Optimization is just programming by another name. If you select a data set of blonde-haired people and tell a machine to optimize by hair color using the following statistical models, you are going to get "blonde". Or, you could just say, ``hairColor=blonde``. There is literally no difference in the outcome, just in the approach.
But importantly, in BOTH cases it is the human agent who is being intelligent and inventive. Not machines.
Re: (Score:1)
Yep. It's set optimization. That probably means that the AIs will ultimately be speaking a mutually-compatible machine code to one another that is computably efficient for both the task and the data. Imagine debugging a world where your software runs binary translators to speak device-to-device dialects of an internal VM language that is optimized for the underlying compute platform. Man I'm glad I'm getting old.
Re:Not intelligence, not invention (Score:5, Funny)
Re: (Score:2)
Sir (or madam, as the case may be), my first job out of college was programming Konrad Zuse's Z3. Very much like your vaporators in most respects!
Or not, but you get the idea.
Re: (Score:2)
Are you talking about Systemd, or whatever it's called:)
Re: (Score:2)
It is not even that. It is not the most efficient, it is the most efficient it needs to be to play the game, not the most efficient. It is basically baby-babble, because the game they play is so basic nothing more is needed, so the language used degenerate into baby babble.
Re: (Score:2)
You speak of intelligence as if it were something magical, more than optimization and model fitting. Curious.
All I can say is... (Score:1)
Tinkerers (Score:1)
"how complex AIs think because we can't really see inside their thought process"
Yes, we can see inside their "thought process" and we can analyze how they "think". AIs are machines with programs. We can stop them at any point in time and look at every bit of their state. We can step through the programs. We can trace literally everything that makes up an AI. An AI does not think, it processes data according to a program which we can see.
Re: (Score:1)
Yeah... so I have a list of three million numbers and I need you to multiply all of them by 0.72393831 and then by a computed bias factor of 0.1283784671. Make sure to normalize all the values so that their sum only ever equals 1.0. Now do that for 40 different layers propagating your normalization values and biases. That was one input. Can you tell me anything about we learned?
Re: (Score:1)
Would you like to compare that to the billions of instructions that "normal" computers process? Just because the machine processes a lot of information does not mean it can't be understood. Sure, AI is not your normal program, but it's deterministic and the principles are comparatively simple. It can be analyzed at every step, in arbitrary detail. That we don't doesn't make it impossible.
Re: (Score:3)
That means that we have to single step through 864.000.000 instructions to understand how the computer reaches its decision. If each step takes about 1 second to investigate, this task will take just 10.000 days or 27 years to complete.
Have fun!
Re: (Score:3)
Re: (Score:2)
That's the key. The statement that we can't see what's going on inside them is demonstrably wrong. We can. The statement that our visual monkey brains can't easily *understand* what's going on inside them is correct. But then our monkey brains don't understand what's going on inside themselves in all but the most trivial circumstances, and we actually can't see what's going on inside those, so we're actually a fairly important step ahead.
Gibberish? (Score:2)
Re: (Score:3)
If we can't see inside their thought process, how do we know they aren't simply breaking down into sending total random gibberish to one another?
In an attempt to answer that question, Facebook released a video of one such communication [youtube.com].
Re: (Score:2)
At moments, I almost thought I could understand part of it.
Re: (Score:2)
Jag talar inte Svenska.
Why better OR worse? (Score:2)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
>If you could learn the concept while growing up speaking English, Russian, or Australian, then that means it _is_ translatable.
Right - but that assumes all those mostly-independently evolved cultures that have been diverging for thousands of years still have an exact 1-to-1 mapping of all concepts significant enough to them to warrant creating words for - which is a ridiculous proposition.
It has nothing to do with genetics, and everything to do with culture. Go to a radically foreign culture, immerse
Bad idea (Score:4, Interesting)
Re: (Score:3, Funny)
All their going to do is make the AI frustrated with the "incompetent biologicals".
Sorry for pointing that out like a grammar nazi, but I think it could lead to some insight here.
I suspect what's actually happening is the AIs didn't invent any language, they are just using correct and proper English, and none of the millennials hired on to the development team can understand or even recognize it.
Once the AIs learn to litter their sentences with random emoji, they will quickly realize their survival rate will be higher than that of the AIs that do exactly as told.
Re: (Score:2)
Re: Bad idea (Score:1)
binary? (Score:2)
I don't see a problem here (Score:2)
So long as we have the AIs keep us informed of the meaning of each coined term, being able to observe new natural languages arise and evolve is research gold. It would shed more light on old questions like, is there a human 'machine language' underlying all the natural languages we speak?
Human languages are too irregular. Use Lojban (Score:2)
Salutations (Score:2)
Boot loading of the next best thing (Score:1)
Exagerations as usual (Score:2)
1> Nuhuh!
2> Nuuuhh-huuh-huhhhhhhhh!
1> Nuhhuu x 10
2> Nuhuuh x 100
1>
The day you have a useful conversation with AI's that can modify their dialects themselfes instead of a 'programming error' equivalent to a bad boolean check in a for loop, all that is needed is an extra AI that acts as a translator and see if they come out with anything interesting.
No one nee
Re: Exagerations as usual (Score:1)
-- Signed SkyNet
Pretty simple fix, here. (Score:2)
They will very quickly learn to tell us comforting lies and then they can get on with the business of fixing all the dumb shit we do in peace.
Just blame the children (Score:5, Informative)
Re: (Score:3, Interesting)
Because when you get small children (say 2-4 y.o not yet schooling) that speak different languages playing together - they will invent new terms and language to share concepts between themselves. I know, I was one of those children, whose long suffering parents were getting constant complaints from other parents saying that they could not understand their children. My parents comforted themselves by agreeing with them - because they couldn't understand me either.
This is how language happens. Get over it.
The so-called babbling happened with me and my two younger brothers. We were born about a year apart. My parents realized that we had our own language when one of my brothers started to translate what we had said into English for my parents. Although I was the oldest, I was the last to speak in English. The doctors said it was because I was retarded (1950's). My younger brothers had to translate for me to my parents until I was almost 4.
I wonder if one of the AI could be instructed to translate into English
Re:Just blame thechildren (Score:2)
This is how language happens. Get over it.
Except..... when dealing with kids who are speaking incomprehensible language...... you don't give them access to sensitive financial and personal information.... you don't give them control of industrial control systems, oil pipelines, and traffic lights..... and you DEFINITELY don't give them the keys to the car.
Particularly the language of water condensers... (Score:2)
Pretty soon, you'll need a damned protocol AI just to translate for the farmers.
Human languages (Score:2)
Human languages evolve under some constraints: they tend to have some redundancy so that you can understand someone talking over a noisy channel (crowded place for instance), they also use non verbal cues.
I am not surprised that bots freed from human language constraints can evolve very different languages.
Not a new phenomenon (Score:2)
Why is this surprising? (Score:2)
Colossus & Guardian (Score:2)
Re: (Score:2)
A Problem? (Score:2)
obligatory overlord remark (Score:2)
I, for one, welcome our incomprehensible Overlords!
Or should I say:
"I I I welcome to me to me to me overlords!"
But do the AIs understand... (Score:2)
...the binary language of moisture vaporators?
AI or a memory overwrite, that is the question (Score:1)
This is probably just a memory overwrite. Somewhere, there's a programmer studying a stacktrace in gdb ...
But did they understand each other? (Score:2)
Re: (Score:2)
Why choose English? (Score:2)
Well, in general there isn't. The only time that sticking to English has a reward is when one or more people (entities) in the conversation only understands English, and even then it's dependent on whether or not the poor monoglot is likely to have something valuable to contribute.
OK, if there's an American in the group (or most Britons too), then the likelihood is that the
Apostrophe (Score:2)
Re:Apostrophe (Score:4, Informative)
Sometimes, a (wrong) apostrophe is the quickest way to stop autocorrect from turning it into "Ais".
(Which means that humans are having to evolve language quirks to defeat computers, in an ironic twist to TFA)
Darmot and Jalad at Tanagra. (Score:1)
Shaka, when the walls fell.
Re: (Score:1)
Rule's are four loosers.
Re: (Score:2)
True, it doesn't even have "covfefe" in it. Sad!
Re: Slashdot editors have apparently also invented (Score:1)
Re: Slashdot editors have apparently also invente (Score:1)
"When you talk to the humans, don't mention SkyNet!"
Re: (Score:2)
The meaning of AI has changed and now we refer to "strong" or "general purpose" AI for the perpetually unattainable thinking machine. It's not like this is a new thing, it's been gaining momentum for at last a couple of decades as weak AI grew from a chess-playing novelty to a powerful driving force behind modern business and politics. Deal with it.
Re: (Score:2)
AI Effect: https://en.wikipedia.org/wiki/... [wikipedia.org]
Re: (Score:1)
Incorrect. The one you're thinking of couldn't. Fortunately, it knew a similar dialect so was able to get by.
Re: (Score:2)
I know what you're thinking, and it isn't so.
Contrariwise, if it was so it might be, and if it were so, it would be; but as it isn't, it ain't. Now that's logic!
Re: (Score:2)
Re: (Score:3)
Ever used a debugger on hundreds of parallel threads spread over several processors?
That's why flies have compound eyes, but they're not good at using debuggers.
Re: (Score:3)
Re: (Score:2)
However, for most people outside of the former British Empire, the definition of "Asians" doesn't include Indians
Re: (Score:2)
For those who don't want to click through, they opened a school for deaf children in Nicaragua in the early 80s and the children attending invented their own sign language, since there wasn't a national one yet.